spark-instrumented-optimizer/sql/core/src/main
Burak Yavuz 6f7ecb0f29 [SPARK-18342] Make rename failures fatal in HDFSBackedStateStore
## What changes were proposed in this pull request?

If the rename operation in the state store fails (`fs.rename` returns `false`), the StateStore should throw an exception and have the task retry. Currently if renames fail, nothing happens during execution immediately. However, you will observe that snapshot operations will fail, and then any attempt at recovery (executor failure / checkpoint recovery) also fails.

## How was this patch tested?

Unit test

Author: Burak Yavuz <brkyvz@gmail.com>

Closes #15804 from brkyvz/rename-state.
2016-11-08 15:08:09 -08:00
..
java/org/apache/spark/sql [SPARK-17830][SQL] Annotate remaining SQL APIs with InterfaceStability 2016-10-13 11:12:30 -07:00
resources [SPARK-16031] Add debug-only socket source in Structured Streaming 2016-06-19 21:27:04 -07:00
scala/org/apache/spark/sql [SPARK-18342] Make rename failures fatal in HDFSBackedStateStore 2016-11-08 15:08:09 -08:00