ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/sql/core
History
Gengliang Wang 6bcd4805d2 [SPARK-27286][SQL] Handles exceptions on proceeding to next record in FilePartitionReader 
## What changes were proposed in this pull request?

In data source V2, the method `PartitionReader.next()` has side effects. When the method is called, the current reader proceeds to the next record.
This might throw RuntimeException/IOException and File source V2 framework should handle these exceptions.

## How was this patch tested?

Unit test.

Closes #24225 from gengliangwang/corruptFile.

Authored-by: Gengliang Wang <gengliang.wang@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2019-03-26 22:33:34 -07:00
..
benchmarks [SPARK-26975][SQL] Support nested-column pruning over limit/sample/repartition 2019-03-19 20:24:22 -07:00
src [SPARK-27286][SQL] Handles exceptions on proceeding to next record in FilePartitionReader 2019-03-26 22:33:34 -07:00
v1.2.1/src [SPARK-27182][SQL] Move the conflict source code of the sql/core module to sql/core/v1.2.1 2019-03-26 22:32:03 -07:00
v2.3.4/src [SPARK-27182][SQL] Move the conflict source code of the sql/core module to sql/core/v1.2.1 2019-03-26 22:32:03 -07:00
pom.xml [SPARK-27182][SQL] Move the conflict source code of the sql/core module to sql/core/v1.2.1 2019-03-26 22:32:03 -07:00