Marcelo Vanzin 6735433cde [SPARK-20079][YARN] Fix client AM not allocating executors after restart. ... The main goal of this change is to avoid the situation described in the bug, where an AM restart in the middle of a job may cause no new executors to be allocated because of faulty logic in the reset path. The change does two things: - fixes the executor alloc manager's reset() so that it does not stop allocation after a reset() in the middle of a job - re-orders the initialization of the YarnAllocator class so that it fetches the current executor ID before triggering the reset() above. This ensures both that the new allocator gets new requests for executors, and that it starts from the correct executor id. Tested with unit tests and by manually causing AM restarts while running jobs using spark-shell in YARN mode. Closes #17882 Author: Marcelo Vanzin <vanzin@cloudera.com> Author: Guoqiang Li <witgo@qq.com> Closes #18663 from vanzin/SPARK-20079.		2017-08-01 10:12:30 -07:00
..
mesos	[SPARK-21502][MESOS] fix --supervise for mesos in cluster mode	2017-07-24 11:11:34 -07:00
yarn	[SPARK-20079][YARN] Fix client AM not allocating executors after restart.	2017-08-01 10:12:30 -07:00