spark-instrumented-optimizer/sql/catalyst/src/main
Nathan Howell c82f16c15e [SPARK-18658][SQL] Write text records directly to a FileOutputStream
## What changes were proposed in this pull request?

This replaces uses of `TextOutputFormat` with an `OutputStream`, which will either write directly to the filesystem or indirectly via a compressor (if so configured). This avoids intermediate buffering.

The inverse of this (reading directly from a stream) is necessary for streaming large JSON records (when `wholeFile` is enabled) so I wanted to keep the read and write paths symmetric.

## How was this patch tested?

Existing unit tests.

Author: Nathan Howell <nhowell@godaddy.com>

Closes #16089 from NathanHowell/SPARK-18658.
2016-12-01 21:40:49 -08:00
..
antlr4/org/apache/spark/sql/catalyst/parser [SPARK-17932][SQL] Support SHOW TABLES EXTENDED LIKE 'identifier_with_wildcards' statement 2016-11-30 03:59:25 -08:00
java/org/apache/spark/sql [SPARK-18053][SQL] compare unsafe and safe complex-type values correctly 2016-11-23 04:15:19 -08:00
scala/org/apache/spark/sql [SPARK-18658][SQL] Write text records directly to a FileOutputStream 2016-12-01 21:40:49 -08:00