spark-instrumented-optimizer/external
dengziming 28ccd31aee [SPARK-29611][WEBUI] Sort Kafka metadata by the number of messages
### What changes were proposed in this pull request?

Sort metadata by the number of messages in each Kafka partition

### Why are the changes needed?

help to find the data skewness problem.

### Does this PR introduce any user-facing change?

Yes, add a column count to the metadata and sort by count
![image](https://user-images.githubusercontent.com/26023240/67617886-63e06800-f81a-11e9-8718-be3a0100952e.png)

If you set `minPartitions` configurations with structure structured-streaming which doesn't have the Streaming page, my code changes in `DirectKafkaInputDStream` won't affect the WEB UI page just as it shows in the follow image

![image](https://user-images.githubusercontent.com/26023240/68020762-79520800-fcda-11e9-96cd-f0c64a36f505.png)

### How was this patch tested?

Manual test

Closes #26266 from dengziming/feature_ui_optimize.

Lead-authored-by: dengziming <dengziming@growingio.com>
Co-authored-by: dengziming <swzmdeng@163.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2019-11-01 22:46:34 -07:00
..
avro Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00
docker [SPARK-28683][BUILD] Upgrade Scala to 2.12.10 2019-09-18 13:30:36 -07:00
docker-integration-tests Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00
kafka-0-10 [SPARK-29611][WEBUI] Sort Kafka metadata by the number of messages 2019-11-01 22:46:34 -07:00
kafka-0-10-assembly Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00
kafka-0-10-sql Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00
kafka-0-10-token-provider Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00
kinesis-asl Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00
kinesis-asl-assembly Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00
spark-ganglia-lgpl Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00