spark-instrumented-optimizer/mllib
Yuhao Yang 8cffcb60de [SPARK-14322][MLLIB] Use treeAggregate instead of reduce in OnlineLDAOptimizer
## What changes were proposed in this pull request?
jira: https://issues.apache.org/jira/browse/SPARK-14322

OnlineLDAOptimizer uses RDD.reduce in two places where it could use treeAggregate. This can cause scalability issues. This should be an easy fix.
This is also a bug since it modifies the first argument to reduce, so we should use aggregate or treeAggregate.
See this line: f12f11e578/mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAOptimizer.scala (L452)
and a few lines below it.

## How was this patch tested?
unit tests

Author: Yuhao Yang <hhbyyh@gmail.com>

Closes #12106 from hhbyyh/ldaTreeReduce.
2016-04-06 11:36:26 -07:00
..
src [SPARK-14322][MLLIB] Use treeAggregate instead of reduce in OnlineLDAOptimizer 2016-04-06 11:36:26 -07:00
pom.xml [SPARK-13715][MLLIB] Remove last usages of jblas in tests 2016-03-08 17:47:55 +00:00