38e4699c97
## What changes were proposed in this pull request? Check on job submit to make sure we don't launch a barrier stage with unsupported RDD chain pattern. The following patterns are not supported: - Ancestor RDDs that have different number of partitions from the resulting RDD (eg. union()/coalesce()/first()/PartitionPruningRDD); - An RDD that depends on multiple barrier RDDs (eg. barrierRdd1.zip(barrierRdd2)). ## How was this patch tested? Add test cases in `BarrierStageOnSubmittedSuite`. Author: Xingbo Jiang <xingbo.jiang@databricks.com> Closes #21927 from jiangxb1987/SPARK-24820. |
||
|---|---|---|
| .. | ||
| java | ||
| resources | ||
| scala/org/apache | ||