2145f07e22
### What changes were proposed in this pull request? Add executor number check. ### Why are the changes needed? The test `decommission workers ensure that shuffle output is regenerated even with shuffle service` assumes it has two executor and both of two tasks can execute concurrently. The two tasks will execute serially if there only one executor. The result is test is unexpceted. E.g. ``` [info] 5 did not equal 4 Expected 4 tasks but got List(0:0:0:0-SUCCESS, 0:0:1:0-FAILED, 0:0:1:1-SUCCESS, 0:1:0:0-SUCCESS, 1:0:0:0-SUCCESS) (DecommissionWorkerSuite.scala:190) ``` The failed task due to the first task finished and decommission the worker. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass exists test. Closes #31255 from ulysses-you/SPARK-34166. Authored-by: ulysses-you <ulyssesyou18@gmail.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |