[SPARK-35311][SS][UI][DOCS] Structured Streaming Web UI state information documentation
### What changes were proposed in this pull request? In this PR I'm adding Structured Streaming Web UI state information documentation. ### Why are the changes needed? Missing documentation. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? ``` cd docs/ SKIP_API=1 bundle exec jekyll build ``` Manual webpage check. Closes #32433 from gaborgsomogyi/SPARK-35311. Authored-by: Gabor Somogyi <gabor.g.somogyi@gmail.com> Signed-off-by: Jungtaek Lim <kabhwan.opensource@gmail.com>
This commit is contained in:
parent
8fa739fb9d
commit
b6a0a7ea53
BIN
docs/img/webui-structured-streaming-detail2.png
Normal file
BIN
docs/img/webui-structured-streaming-detail2.png
Normal file
Binary file not shown.
After Width: | Height: | Size: 353 KiB |
|
@ -415,6 +415,7 @@ click a "run id" in the tables.
|
|||
|
||||
<p style="text-align: center;">
|
||||
<img src="img/webui-structured-streaming-detail.png" title="Structured Streaming Query Statistics" alt="Structured Streaming Query Statistics">
|
||||
<img src="img/webui-structured-streaming-detail2.png">
|
||||
</p>
|
||||
|
||||
The statistics page displays some useful metrics for insight into the status of your streaming
|
||||
|
@ -431,6 +432,11 @@ The tracked operations are listed as follows.
|
|||
* latestOffset & getOffset: Time taken to query the maximum available offset for this source.
|
||||
* queryPlanning: Time taken to generates the execution plan.
|
||||
* walCommit: Time taken to write the offsets to the metadata log.
|
||||
* **Global Watermark Gap.** The gap between batch timestamp and global watermark for the batch.
|
||||
* **Aggregated Number Of Total State Rows.** The aggregated number of total state rows.
|
||||
* **Aggregated Number Of Updated State Rows.** The aggregated number of updated state rows.
|
||||
* **Aggregated State Memory Used In Bytes.** The aggregated state memory used in bytes.
|
||||
* **Aggregated Number Of State Rows Dropped By Watermark.** The aggregated number of state rows dropped by watermark.
|
||||
|
||||
As an early-release version, the statistics page is still under development and will be improved in
|
||||
future releases.
|
||||
|
|
Loading…
Reference in a new issue