24d39a5ee2
### What changes were proposed in this pull request? Add check if `CoalesceShufflePartitions` really coalesce shuffle partition number. ### Why are the changes needed? The `CoalesceShufflePartitions` can not coalesce such case if the total shuffle partitions size of mappers are big enough. Then it's confused to use `CustomShuffleReaderExec` which marked as `coalesced` but has no affect with partition number. ### Does this PR introduce _any_ user-facing change? Probably yes, the plan changed. ### How was this patch tested? Add test. Closes #31994 from ulysses-you/SPARK-34899. Authored-by: ulysses-you <ulyssesyou18@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |