spark-instrumented-optimizer/core
Sebastien Rainville eb019af9a9 [SPARK-13001][CORE][MESOS] Prevent getting offers when reached max cores
Similar to https://github.com/apache/spark/pull/8639

This change rejects offers for 120s when reached `spark.cores.max` in coarse-grained mode to mitigate offer starvation. This prevents Mesos to send us offers again and again, starving other frameworks. This is especially problematic when running many small frameworks on the same Mesos cluster, e.g. many small Sparks streaming jobs, and cause the bigger spark jobs to stop receiving offers. By rejecting the offers for a long period of time, they become available to those other frameworks.

Author: Sebastien Rainville <sebastien@hopper.com>

Closes #10924 from sebastienrainville/master.
2016-05-04 14:32:36 -07:00
..
src [SPARK-13001][CORE][MESOS] Prevent getting offers when reached max cores 2016-05-04 14:32:36 -07:00
pom.xml Revert "[SPARK-14613][ML] Add @Since into the matrix and vector classes in spark-mllib-local" 2016-04-28 19:57:41 -07:00