ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/docs/img
History
Luca Canali b5df40bd87 [SPARK-29894][SQL][WEBUI] Add Codegen Stage Id to Spark plan graphs in Web UI SQL Tab 
### What changes were proposed in this pull request?
The Web UI SQL Tab provides information on the executed SQL using plan graphs and by reporting SQL execution plans. Both sources provide useful information. Physical execution plans report Codegen Stage Ids. This PR adds Codegen Stage Ids to the plan graphs.

### Why are the changes needed?
It is useful to have Codegen Stage Id information also reported in plan graphs, this allows to more easily match physical plans and graphs with metrics when troubleshooting SQL execution.
Example snippet to show the proposed change:

![](https://issues.apache.org/jira/secure/attachment/12985837/snippet__plan_graph_with_Codegen_Stage_Id_Annotated.png)

Example of the current state:
![](https://issues.apache.org/jira/secure/attachment/12985838/snippet_plan_graph_before_patch.png)

Physical plan:
![](https://issues.apache.org/jira/secure/attachment/12985932/Physical_plan_Annotated.png)

### Does this PR introduce any user-facing change?
This PR adds Codegen Stage Id information to SQL plan graphs in the Web UI/SQL Tab.

### How was this patch tested?
Added a test + manually tested

Closes #26519 from LucaCanali/addCodegenStageIdtoWEBUIGraphs.

Authored-by: Luca Canali <luca.canali@cern.ch>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2019-11-20 23:20:33 +08:00
..
AllJobsPageDetail1.png [SPARK-28543][DOCS][WEBUI] Document Spark Jobs page 2019-08-15 08:52:23 -05:00
AllJobsPageDetail2.png [SPARK-28543][DOCS][WEBUI] Document Spark Jobs page 2019-08-15 08:52:23 -05:00
AllJobsPageDetail3.png [SPARK-28543][DOCS][WEBUI] Document Spark Jobs page 2019-08-15 08:52:23 -05:00
AllStagesPageDetail1.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
AllStagesPageDetail2.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
AllStagesPageDetail3.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
AllStagesPageDetail4.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
AllStagesPageDetail5.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
AllStagesPageDetail6.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
AllStagesPageDetail7.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
AllStagesPageDetail8.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
AllStagesPageDetail9.png [SPARK-28542][DOCS][WEBUI] Stages Tab 2019-08-31 13:33:44 -05:00
cluster-overview.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
cluster-overview.pptx [SPARK-6343] Doc driver-worker network reqs 2015-04-09 06:37:20 -04:00
edge_cut_vs_vertex_cut.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
glyphicons-halflings-white.png Adding docs directory containing documentation currently on the wiki 2012-09-12 13:03:43 -07:00
glyphicons-halflings.png Adding docs directory containing documentation currently on the wiki 2012-09-12 13:03:43 -07:00
graphx_figures.pptx Finished docummenting join operators and revised some of the initial presentation. 2014-01-11 13:48:35 -08:00
graphx_logo.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
java-sm.png Merge pull request #497 from tdas/docs-update 2014-01-28 21:51:05 -08:00
JDBCServer1.png [SPARK-28373][DOCS][WEBUI] JDBC/ODBC Server Tab 2019-09-14 10:18:52 -07:00
JDBCServer2.png [SPARK-28373][DOCS][WEBUI] JDBC/ODBC Server Tab 2019-09-14 10:18:52 -07:00
JDBCServer3.png [SPARK-28373][DOCS][WEBUI] JDBC/ODBC Server Tab 2019-09-14 10:18:52 -07:00
JobPageDetail1.png [SPARK-28543][DOCS][WEBUI] Document Spark Jobs page 2019-08-15 08:52:23 -05:00
JobPageDetail2.png [SPARK-28543][DOCS][WEBUI] Document Spark Jobs page 2019-08-15 08:52:23 -05:00
JobPageDetail3.png [SPARK-28543][DOCS][WEBUI] Document Spark Jobs page 2019-08-15 08:52:23 -05:00
k8s-cluster-mode.png [SPARK-22648][K8S] Spark on Kubernetes - Documentation 2017-12-21 17:21:11 -08:00
ml-Pipeline.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
ml-PipelineModel.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
ml-Pipelines.pptx [SPARK-4575] [mllib] [docs] spark.ml pipelines doc + bug fixes 2014-12-04 17:00:06 +08:00
property_graph.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
python-sm.png Merge pull request #497 from tdas/docs-update 2014-01-28 21:51:05 -08:00
scala-sm.png Merge pull request #497 from tdas/docs-update 2014-01-28 21:51:05 -08:00
spark-logo-hd.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
spark-webui-accumulators.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
streaming-arch.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
streaming-dstream-ops.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
streaming-dstream-window.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
streaming-dstream.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
streaming-figures.pptx [SPARK-2419][Streaming][Docs] More updates to the streaming programming guide 2014-09-06 14:46:43 -07:00
streaming-flow.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
streaming-kinesis-arch.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
structured-streaming-example-model.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
structured-streaming-late-data.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
structured-streaming-model.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
structured-streaming-stream-as-a-table.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
structured-streaming-watermark-append-mode.png [SPARK-19074][SS][DOCS] Updated Structured Streaming Programming Guide for update mode and source/sink options 2017-01-06 11:29:01 -08:00
structured-streaming-watermark-update-mode.png [SPARK-19074][SS][DOCS] Updated Structured Streaming Programming Guide for update mode and source/sink options 2017-01-06 11:29:01 -08:00
structured-streaming-window.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
structured-streaming.pptx [SPARK-19074][SS][DOCS] Updated Structured Streaming Programming Guide for update mode and source/sink options 2017-01-06 11:29:01 -08:00
triplet.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
vertex_routing_edge_tables.png [MINOR][DOCS] Remove unused images; crush PNGs that could use it for good measure 2016-07-04 09:21:58 +01:00
webui-env-class.png [SPARK-28540][WEBUI] Document Environment page 2019-08-21 10:48:48 -05:00
webui-env-hadoop.png [SPARK-28540][WEBUI] Document Environment page 2019-08-21 10:48:48 -05:00
webui-env-sys.png [SPARK-28540][WEBUI] Document Environment page 2019-08-21 10:48:48 -05:00
webui-env-tab.png [SPARK-28540][WEBUI] Document Environment page 2019-08-21 10:48:48 -05:00
webui-exe-err.png [SPARK-28539][WEBUI][DOC] Document Executors page 2019-08-28 08:34:24 -05:00
webui-exe-tab.png [SPARK-28539][WEBUI][DOC] Document Executors page 2019-08-28 08:34:24 -05:00
webui-exe-thread.png [SPARK-28539][WEBUI][DOC] Document Executors page 2019-08-28 08:34:24 -05:00
webui-sql-dag.png [SPARK-29894][SQL][WEBUI] Add Codegen Stage Id to Spark plan graphs in Web UI SQL Tab 2019-11-20 23:20:33 +08:00
webui-sql-plan.png [SPARK-28538][UI] Document SQL page 2019-08-12 08:36:01 -05:00
webui-sql-tab.png [SPARK-28539][WEBUI][DOC] Document Executors page 2019-08-28 08:34:24 -05:00
webui-storage-detail.png [SPARK-28541][WEBUI] Document Storage page 2019-08-20 20:05:13 -05:00
webui-storage-tab.png [SPARK-28541][WEBUI] Document Storage page 2019-08-20 20:05:13 -05:00