799e13161e
## What changes were proposed in this pull request? A shuffle service can serves blocks from multiple apps/tasks. Thus the shuffle service can suffers high memory usage when lots of shuffle-reads happen at the same time. In my cluster, OOM always happens on shuffle service. Analyzing heap dump, memory cost by Netty(ChannelOutboundBufferEntry) can be up to 2~3G. It might make sense to reject "open blocks" request when memory usage is high on shuffle service. |
||
---|---|---|
.. | ||
kvstore | ||
network-common | ||
network-shuffle | ||
network-yarn | ||
sketch | ||
tags | ||
unsafe |