spark-instrumented-optimizer/core/src/test
Xingbo Jiang 3c2fc19d47 [SPARK-18294][CORE] Implement commit protocol to support mapred package's committer
## What changes were proposed in this pull request?

This PR makes the following changes:

- Implement a new commit protocol `HadoopMapRedCommitProtocol` which support the old `mapred` package's committer;
- Refactor SparkHadoopWriter and SparkHadoopMapReduceWriter, now they are combined together, thus we can support write through both mapred and mapreduce API by the new SparkHadoopWriter, a lot of duplicated codes are removed.

After this change, it should be pretty easy for us to support the committer from both the new and the old hadoop API at high level.

## How was this patch tested?
No major behavior change, passed the existing test cases.

Author: Xingbo Jiang <xingbo.jiang@databricks.com>

Closes #18438 from jiangxb1987/SparkHadoopWriter.
2017-06-30 20:30:26 +08:00
..
java [SPARK-3577] Report Spill size on disk for UnsafeExternalSorter 2017-06-29 14:25:51 +08:00
resources [SPARK-19937] Collect metrics for remote bytes read to disk during shuffle. 2017-06-22 14:10:51 -07:00
scala/org/apache [SPARK-18294][CORE] Implement commit protocol to support mapred package's committer 2017-06-30 20:30:26 +08:00