79e148ee93
### What changes were proposed in this pull request?
Disable push-based shuffle when IO encryption is enabled or serializer does not support relocation of serialized objects.
### Why are the changes needed?
Push based shuffle is not compatible with IO encryption or non-relocatable serialization.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Added some tests to check whether push-based shuffle can be disabled successfully when IO encryption is enabled or a serializer that does not support relocation of serialized object is used.
Closes #33976 from rmcyang/SPARK-36705.
Authored-by: Minchu Yang <minyang@minyang-mn3.linkedin.biz>
Signed-off-by: Mridul Muralidharan <mridul<at>gmail.com>
(cherry picked from commit
|
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |