liupengcheng 52a180f25f [SPARK-26674][CORE] Consolidate CompositeByteBuf when reading large frame ... ## What changes were proposed in this pull request? Currently, TransportFrameDecoder will not consolidate the buffers read from network which may cause memory waste. Actually, bytebuf's writtenIndex is far less than it's capacity in most cases, so we can optimize it by doing consolidation. This PR will do this optimization. Related codes: 9a30e23211/common/network-common/src/main/java/org/apache/spark/network/util/TransportFrameDecoder.java (L143) ## How was this patch tested? UT Please review http://spark.apache.org/contributing.html before opening a pull request. Closes #23602 from liupc/Reduce-memory-consumption-in-TransportFrameDecoder. Lead-authored-by: liupengcheng <liupengcheng@xiaomi.com> Co-authored-by: Liupengcheng <liupengcheng@xiaomi.com> Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>		2019-02-25 16:40:46 -08:00
..
kvstore	[SPARK-26090][CORE][SQL][ML] Resolve most miscellaneous deprecation and build warnings for Spark 3	2018-11-19 09:16:42 -06:00
network-common	[SPARK-26674][CORE] Consolidate CompositeByteBuf when reading large frame	2019-02-25 16:40:46 -08:00
network-shuffle	[SPARK-26604][CORE] Clean up channel registration for StreamManager	2019-01-16 10:58:07 +08:00
network-yarn	[SPARK-25642][YARN] Adding two new metrics to record the number of registered connections as well as the number of active connections to YARN Shuffle Service	2018-12-21 11:28:33 -08:00
sketch	[SPARK-25956] Make Scala 2.12 as default Scala version in Spark 3.0	2018-11-14 16:22:23 -08:00
tags	[SPARK-26026][BUILD] Published Scaladoc jars missing from Maven Central	2018-11-19 08:06:33 -06:00
unsafe	[CORE][MINOR] Fix some typos about MemoryMode	2019-01-15 14:40:00 +08:00