[SPARK-32920][FOLLOW-UP][CORE] Shutdown shuffleMergeFinalizeScheduler when DAGScheduler stop
### What changes were proposed in this pull request? Call `shuffleMergeFinalizeScheduler.shutdownNow()` in `DAGScheduler.stop()`. ### Why are the changes needed? Avoid the thread leak. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass existing tests. Closes #33495 from Ngone51/SPARK-32920-followup. Authored-by: yi.wu <yi.wu@databricks.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This commit is contained in:
parent
70a15868fc
commit
21450b3254
|
@ -2512,6 +2512,7 @@ private[spark] class DAGScheduler(
|
|||
|
||||
def stop(): Unit = {
|
||||
messageScheduler.shutdownNow()
|
||||
shuffleMergeFinalizeScheduler.shutdownNow()
|
||||
eventProcessLoop.stop()
|
||||
taskScheduler.stop()
|
||||
}
|
||||
|
|
Loading…
Reference in a new issue