spark-instrumented-optimizer/common
Liang-Chi Hsieh cf133e6110 [SPARK-26604][CORE] Clean up channel registration for StreamManager
## What changes were proposed in this pull request?

Now in `TransportRequestHandler.processStreamRequest`, when a stream request is processed, the stream id is not registered with the current channel in stream manager. It should do that so in case of that the channel gets terminated we can remove associated streams of stream requests too.

This also cleans up channel registration in `StreamManager`. Since `StreamManager` doesn't register channel but only `OneForOneStreamManager` does it, this removes `registerChannel` from `StreamManager`. When `OneForOneStreamManager` goes to register stream, it will also register channel for the stream.

## How was this patch tested?

Existing tests.

Closes #23521 from viirya/SPARK-26604.

Authored-by: Liang-Chi Hsieh <viirya@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2019-01-16 10:58:07 +08:00
..
kvstore [SPARK-26090][CORE][SQL][ML] Resolve most miscellaneous deprecation and build warnings for Spark 3 2018-11-19 09:16:42 -06:00
network-common [SPARK-26604][CORE] Clean up channel registration for StreamManager 2019-01-16 10:58:07 +08:00
network-shuffle [SPARK-26604][CORE] Clean up channel registration for StreamManager 2019-01-16 10:58:07 +08:00
network-yarn [SPARK-25642][YARN] Adding two new metrics to record the number of registered connections as well as the number of active connections to YARN Shuffle Service 2018-12-21 11:28:33 -08:00
sketch [SPARK-25956] Make Scala 2.12 as default Scala version in Spark 3.0 2018-11-14 16:22:23 -08:00
tags [SPARK-26026][BUILD] Published Scaladoc jars missing from Maven Central 2018-11-19 08:06:33 -06:00
unsafe [CORE][MINOR] Fix some typos about MemoryMode 2019-01-15 14:40:00 +08:00