zhoukang 4b88393cb9 [SPARK-21922] Fix duration always updating when task failed but status is still RUN… ... …NING ## What changes were proposed in this pull request? When driver quit abnormally which cause executor shutdown and task metrics can not be sent to driver for updating.In this case the status will always be 'RUNNING' and the duration on history UI will be 'CurrentTime - launchTime' which increase infinitely. We can fix this time by modify time of event log since this time has gotten when `FSHistoryProvider` fetch event log from File System. And the result picture is uploaded in [SPARK-21922](https://issues.apache.org/jira/browse/SPARK-21922). How to reproduce? (1) Submit a job to spark on yarn (2) Mock an oom(or other case can make driver quit abnormally) senario for driver (3) Make sure executor is running task when driver quitting (4) Open the history server and checkout result It is not a corner case since there are many such jobs in our current cluster. ## How was this patch tested? Deploy historyserver and open a job has this problem. Author: zhoukang <zhoukang199191@gmail.com> Closes #19132 from caneGuy/zhoukang/fix-duration.		2017-09-14 20:40:33 +08:00
..
src	[SPARK-21922] Fix duration always updating when task failed but status is still RUN…	2017-09-14 20:40:33 +08:00
pom.xml	[SPARK-21276][CORE] Update lz4-java to the latest (v1.4.0)	2017-08-09 17:31:52 +02:00