From fa0c7f487b67ec94f40b0fc914b6a8efff4eae8d Mon Sep 17 00:00:00 2001 From: Gengliang Wang Date: Wed, 28 Jul 2021 13:54:35 +0800 Subject: [PATCH] [SPARK-34399][DOCS][FOLLOWUP] Add docs for the new metrics of task/job commit time ### What changes were proposed in this pull request? This is follow-up of https://github.com/apache/spark/pull/31522. It adds docs for the new metrics of task/job commit time ### Why are the changes needed? So that users can understand the metrics better and know that the new metrics are only for file table writes. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Build docs and preview: ![image](https://user-images.githubusercontent.com/1097932/127198210-2ab201d3-5fca-4065-ace6-0b930390380f.png) Closes #33542 from gengliangwang/addDocForMetrics. Authored-by: Gengliang Wang Signed-off-by: Wenchen Fan (cherry picked from commit c9a7ff3f36838fad5b62fa5d9be020aa465e4193) Signed-off-by: Wenchen Fan --- docs/web-ui.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/web-ui.md b/docs/web-ui.md index deaf50fe57..ec86945ca2 100644 --- a/docs/web-ui.md +++ b/docs/web-ui.md @@ -404,7 +404,8 @@ Here is the list of SQL metrics: avg hash probe bucket list iters the average bucket list iterations per lookup during aggregation HashAggregate data size of build side the size of built hash map ShuffledHashJoin time to build hash map the time spent on building hash map ShuffledHashJoin - + task commit time the time spent on committing the output of a task after the writes succeed any write operation on a file-based table + job commit time the time spent on committing the output of a job after the writes succeed any write operation on a file-based table ## Structured Streaming Tab