spark-instrumented-optimizer/common
Ryan Blue 67e085ef6d [SPARK-16420] Ensure compression streams are closed.
## What changes were proposed in this pull request?

This uses the try/finally pattern to ensure streams are closed after use. `UnsafeShuffleWriter` wasn't closing compression streams, causing them to leak resources until garbage collected. This was causing a problem with codecs that use off-heap memory.

## How was this patch tested?

Current tests are sufficient. This should not change behavior.

Author: Ryan Blue <blue@apache.org>

Closes #14093 from rdblue/SPARK-16420-unsafe-shuffle-writer-leak.
2016-07-08 12:37:26 -07:00
..
network-common [SPARK-16420] Ensure compression streams are closed. 2016-07-08 12:37:26 -07:00
network-shuffle [SPARK-15263][CORE] Make shuffle service dir cleanup faster by using rm -rf 2016-05-18 12:10:32 +01:00
network-yarn [SPARK-16018][SHUFFLE] Shade netty to load shuffle jar in Nodemanger 2016-06-17 15:44:33 -05:00
sketch [SPARK-15290][BUILD] Move annotations, like @Since / @DeveloperApi, into spark-tags 2016-05-17 09:55:53 +01:00
tags [SPARK-15290][BUILD] Move annotations, like @Since / @DeveloperApi, into spark-tags 2016-05-17 09:55:53 +01:00
unsafe [SPARK-16021] Fill freed memory in test to help catch correctness bugs 2016-07-06 16:30:25 -07:00