spark-instrumented-optimizer/network
Lianhui Wang 6e5fc37883 [SPARK-11252][NETWORK] ShuffleClient should release connection after fetching blocks had been completed for external shuffle
with yarn's external shuffle, ExternalShuffleClient of executors reserve its connections for yarn's NodeManager until application has been completed. so it will make NodeManager and executors have many socket connections.
in order to reduce network pressure of NodeManager's shuffleService, after registerWithShuffleServer or fetchBlocks have been completed in ExternalShuffleClient, connection for NM's shuffleService needs to be closed.andrewor14 rxin vanzin

Author: Lianhui Wang <lianhuiwang09@gmail.com>

Closes #9227 from lianhuiwang/spark-11252.
2015-11-10 10:40:08 -08:00
..
common [SPARK-11252][NETWORK] ShuffleClient should release connection after fetching blocks had been completed for external shuffle 2015-11-10 10:40:08 -08:00
shuffle [SPARK-11252][NETWORK] ShuffleClient should release connection after fetching blocks had been completed for external shuffle 2015-11-10 10:40:08 -08:00
yarn [SPARK-10997][CORE] Add "client mode" to netty rpc env. 2015-11-02 10:26:36 -08:00