ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/sql/hive
History
Cheng Lian d8a53fb806 [SPARK-8079] [SQL] Makes InsertIntoHadoopFsRelation job/task abortion more robust 
As described in SPARK-8079, when writing a DataFrame to a `HadoopFsRelation`, if `HadoopFsRelation.prepareForWriteJob` throws exception, an unexpected NPE will be thrown during job abortion. (This issue doesn't bring much damage since the job is failing anyway.)

This PR makes the job/task abortion logic in `InsertIntoHadoopFsRelation` more robust to avoid such confusing exceptions.

Author: Cheng Lian <lian@databricks.com>

Closes #6612 from liancheng/spark-8079 and squashes the following commits:

87cd81e [Cheng Lian] Addresses @rxin's comment
1864c75 [Cheng Lian] Addresses review comments
9e6dbb3 [Cheng Lian] Makes InsertIntoHadoopFsRelation job/task abortion more robust

(cherry picked from commit 16fc49617e)
Signed-off-by: Cheng Lian <lian@databricks.com>
2015-06-06 17:23:46 +08:00
..
compatibility/src/test/scala/org/apache/spark/sql/hive/execution [SQL] [TEST] udf_java_method failed due to jdk version 2015-05-21 12:32:10 -07:00
src [SPARK-8079] [SQL] Makes InsertIntoHadoopFsRelation job/task abortion more robust 2015-06-06 17:23:46 +08:00
v0.13.1/src/main/scala/org/apache/spark/sql/hive [SPARK-6505] [SQL] Remove the reflection call in HiveFunctionWrapper 2015-04-27 14:08:05 +08:00
pom.xml [SPARK-7558] Demarcate tests in unit-tests.log (1.4) 2015-06-03 20:46:44 -07:00