Mark Hamstra 0a5aef753e [SPARK-10666][SPARK-6880][CORE] Use properties from ActiveJob associated with a Stage ... This issue was addressed in https://github.com/apache/spark/pull/5494, but the fix in that PR, while safe in the sense that it will prevent the SparkContext from shutting down, misses the actual bug. The intent of `submitMissingTasks` should be understood as "submit the Tasks that are missing for the Stage, and run them as part of the ActiveJob identified by jobId". Because of a long-standing bug, the `jobId` parameter was never being used. Instead, we were trying to use the jobId with which the Stage was created -- which may no longer exist as an ActiveJob, hence the crash reported in SPARK-6880. The correct fix is to use the ActiveJob specified by the supplied jobId parameter, which is guaranteed to exist at the call sites of submitMissingTasks. This fix should be applied to all maintenance branches, since it has existed since 1.0. kayousterhout pankajarora12 Author: Mark Hamstra <markhamstra@gmail.com> Author: Imran Rashid <irashid@cloudera.com> Closes #6291 from markhamstra/SPARK-6880.		2015-11-25 09:34:34 -06:00
..
java	[SPARK-8029] Robust shuffle writer	2015-11-12 22:44:57 -08:00
resources	[SPARK-8673] [LAUNCHER] API and infrastructure for communicating with child apps.	2015-10-09 15:28:09 -05:00
scala/org/apache	[SPARK-10666][SPARK-6880][CORE] Use properties from ActiveJob associated with a Stage	2015-11-25 09:34:34 -06:00