ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-29763] Fix Stage UI Page not showing all accumulators in Task Table

Browse source 
### What changes were proposed in this pull request?
Fix the task table UI to show all accumulators.

Below example was creating 2 accumulators
scala> val accum = sc.longAccumulator("My Accumulator")
scala> val accum2 = sc.longAccumulator("My Accumulator")
scala> sc.parallelize(Array(1, 2, 3, 4)).foreach(x => {
     accum2.add(x)
     accum.add(x)
     })

Before this change, only shows a single on in task table:
![beforefixtaskui](https://user-images.githubusercontent.com/4563792/68225858-b0fcd080-ffb6-11e9-8561-3dc25a81a106.png)

After this change you can see all of them:
![tasktablegood](https://user-images.githubusercontent.com/4563792/68225911-c5d96400-ffb6-11e9-952a-18d3738711d1.png)

### Why are the changes needed?

Its not showing all accumulators now.

### Does this PR introduce any user-facing change?

no

### How was this patch tested?

Manual testing the UI.

Closes #26402 from tgravescs/SPARK-29763.

Authored-by: Thomas Graves <tgraves@nvidia.com>
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
This commit is contained in:
Thomas Graves 2019-11-05 14:15:14 -08:00 committed by Marcelo Vanzin
parent 4c53ac1822
commit 075cd557f1
1 changed files with 5 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
core/src/main/resources/org/apache/spark/ui/static/stagepage.js
View file

@ -758,8 +758,11 @@ $(document).ready(function () {
{
data : function (row, type) {
if (accumulatorTable.length > 0 && row.accumulatorUpdates.length > 0) {
var accIndex = row.accumulatorUpdates.length - 1;
return row.accumulatorUpdates[accIndex].name + ' : ' + row.accumulatorUpdates[accIndex].update;
var allAccums = "";
row.accumulatorUpdates.forEach(function(accumulator) {
allAccums += accumulator.name + ': ' + accumulator.update + "<BR>";
})
return allAccums;
} else {
return "";
}