SPARK-4300 [CORE] Race condition during SparkWorker shutdown

Close appender saving stdout/stderr before destroying process to avoid exception on reading closed input stream.
(This also removes a redundant `waitFor()` although it was harmless)

CC tdas since I think you wrote this method.

Author: Sean Owen <sowen@cloudera.com>

Closes #4787 from srowen/SPARK-4300 and squashes the following commits:

e0cdabf [Sean Owen] Close appender saving stdout/stderr before destroying process to avoid exception on reading closed input stream
This commit is contained in:
Sean Owen 2015-02-26 14:08:56 -08:00 committed by Andrew Or
parent 5f3238b3b0
commit 3fb53c0298

View file

@ -85,14 +85,13 @@ private[spark] class ExecutorRunner(
var exitCode: Option[Int] = None
if (process != null) {
logInfo("Killing process!")
process.destroy()
process.waitFor()
if (stdoutAppender != null) {
stdoutAppender.stop()
}
if (stderrAppender != null) {
stderrAppender.stop()
}
process.destroy()
exitCode = Some(process.waitFor())
}
worker ! ExecutorStateChanged(appId, execId, state, message, exitCode)