ee5a5a0925
## What changes were proposed in this pull request? Removes check that `spark.executor.instances` is set to 0 when using Streaming DRA. ## How was this patch tested? Manual tests My only concern with this PR is that `spark.executor.instances` (or the actual initial number of executors that the cluster manager gives Spark) can be outside of `spark.streaming.dynamicAllocation.minExecutors` to `spark.streaming.dynamicAllocation.maxExecutors`. I don't see a good way around that, because this code only runs after the SparkContext has been created. Author: Karthik Palaniappan <karthikpal@google.com> Closes #19183 from karth295/master. |
||
---|---|---|
.. | ||
main | ||
test |