0099715aae
### What changes were proposed in this pull request? Fix the setting issue of shuffle batch fetch in `ShuffledRowRDD`. ### Why are the changes needed? Currently, we can not disable the shuffle batch fetch mode once the batch fetch mode has been enabled. This PR fixes the issue to make `ShuffledRowRDD` respects the `spark.sql.adaptive.fetchShuffleBlocksInBatch` at runtime. ### Does this PR introduce _any_ user-facing change? Yes. Before this PR, users can not disable batch fetch if they enabled first. After this PR, they can. ### How was this patch tested? Added unit test. Closes #31155 from Ngone51/fix-batchfetch-set-issue. Authored-by: yi.wu <yi.wu@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |