spark-instrumented-optimizer/core
Aaron Davidson 0a14421765 SPARK-1700: Close socket file descriptors on task completion
This will ensure that sockets do not build up over the course of a job, and that cancellation successfully cleans up sockets.

Tested in standalone mode. More file descriptors spawn than expected (around 1000ish rather than the expected 8ish) but they do not pile up between runs, or as high as before (where they went up to around 5k).

Author: Aaron Davidson <aaron@databricks.com>

Closes #623 from aarondav/pyspark2 and squashes the following commits:

0ca13bb [Aaron Davidson] SPARK-1700: Close socket file descriptors on task completion
2014-05-02 23:55:13 -07:00
..
src SPARK-1700: Close socket file descriptors on task completion 2014-05-02 23:55:13 -07:00
pom.xml SPARK-1004. PySpark on YARN 2014-04-29 23:24:34 -07:00