spark-instrumented-optimizer/network
Josh Rosen 5f37aad48c [SPARK-13308] ManagedBuffers passed to OneToOneStreamManager need to be freed in non-error cases
ManagedBuffers that are passed to `OneToOneStreamManager.registerStream` need to be freed by the manager once it's done using them. However, the current code only frees them in certain error-cases and not during typical operation. This isn't a major problem today, but it will cause memory leaks after we implement better locking / pinning in the BlockManager (see #10705).

This patch modifies the relevant network code so that the ManagedBuffers are freed as soon as the messages containing them are processed by the lower-level Netty message sending code.

/cc zsxwing for review.

Author: Josh Rosen <joshrosen@databricks.com>

Closes #11193 from JoshRosen/add-missing-release-calls-in-network-layer.
2016-02-16 12:06:30 -08:00
..
common [SPARK-13308] ManagedBuffers passed to OneToOneStreamManager need to be freed in non-error cases 2016-02-16 12:06:30 -08:00
shuffle [SPARK-6363][BUILD] Make Scala 2.11 the default Scala version 2016-01-30 00:20:28 -08:00
yarn [SPARK-12807][YARN] Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3 2016-02-09 11:01:47 -08:00