ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/core/src/main
History
Kay Ousterhout a2d33d0b01 [SPARK-5701] Only set ShuffleReadMetrics when task has shuffle deps 
The updateShuffleReadMetrics method in TaskMetrics (called by the executor heartbeater) will currently always add a ShuffleReadMetrics to TaskMetrics (with values set to 0), even when the task didn't read any shuffle data. ShuffleReadMetrics should only be added if the task reads shuffle data.

Author: Kay Ousterhout <kayousterhout@gmail.com>

Closes #4488 from kayousterhout/SPARK-5701 and squashes the following commits:

673ed58 [Kay Ousterhout] SPARK-5701: Only set ShuffleReadMetrics when task has shuffle deps
2015-02-09 21:22:09 -08:00
..
java/org/apache/spark [SPARK-5549] Define TaskContext interface in Scala. 2015-02-03 00:46:04 -08:00
resources/org/apache/spark [SPARK-5388] Provide a stable application submission gateway for standalone cluster mode 2015-02-06 15:57:06 -08:00
scala/org/apache/spark [SPARK-5701] Only set ShuffleReadMetrics when task has shuffle deps 2015-02-09 21:22:09 -08:00