Reynold Xin ef29a9a9aa [SPARK-4307] Initialize FileDescriptor lazily in FileRegion. ... Netty's DefaultFileRegion requires a FileDescriptor in its constructor, which means we need to have a opened file handle. In super large workloads, this could lead to too many open files due to the way these file descriptors are cleaned. This pull request creates a new LazyFileRegion that initializes the FileDescriptor when we are sending data for the first time. Author: Reynold Xin <rxin@databricks.com> Author: Reynold Xin <rxin@apache.org> Closes #3172 from rxin/lazyFD and squashes the following commits: 0bdcdc6 [Reynold Xin] Added reference to Netty's DefaultFileRegion d4564ae [Reynold Xin] Added SparkConf to the ctor argument of IndexShuffleBlockManager. 6ed369e [Reynold Xin] Code review feedback. 04cddc8 [Reynold Xin] [SPARK-4307] Initialize FileDescriptor lazily in FileRegion.		2014-11-11 00:25:31 -08:00
..
java/org/apache/spark	[SPARK-2759][CORE] Generic Binary File Support in Spark	2014-11-01 11:59:39 -07:00
resources	[SPARK-3748] Log thread name in unit test logs	2014-10-01 01:03:49 -07:00
scala/org/apache/spark	[SPARK-4307] Initialize FileDescriptor lazily in FileRegion.	2014-11-11 00:25:31 -08:00