spark-instrumented-optimizer/core/src/test
Xingbo Jiang 5059255d91 [SPARK-25161][CORE] Fix several bugs in failure handling of barrier execution mode
## What changes were proposed in this pull request?

Fix several bugs in failure handling of barrier execution mode:
* Mark TaskSet for a barrier stage as zombie when a task attempt fails;
* Multiple barrier task failures from a single barrier stage should not trigger multiple stage retries;
* Barrier task failure from a previous failed stage attempt should not trigger stage retry;
* Fail the job when a task from a barrier ResultStage failed;
* RDD.isBarrier() should not rely on `ShuffleDependency`s.

## How was this patch tested?

Added corresponding test cases in `DAGSchedulerSuite` and `TaskSchedulerImplSuite`.

Closes #22158 from jiangxb1987/failure.

Authored-by: Xingbo Jiang <xingbo.jiang@databricks.com>
Signed-off-by: Xiangrui Meng <meng@databricks.com>
2018-08-21 08:25:02 -07:00
..
java [SPARK-19355][SQL] Use map output statistics to improve global limit's parallelism 2018-08-10 11:32:15 +02:00
resources [SPARK-23189][CORE][WEB UI] Reflect stage level blacklisting on executor tab 2018-02-13 09:54:52 -06:00
scala/org/apache [SPARK-25161][CORE] Fix several bugs in failure handling of barrier execution mode 2018-08-21 08:25:02 -07:00