ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[MLlib] Update SVD documentation in IndexedRowMatrix

Browse source 
Updating this to reflect the newest SVD via ARPACK

Author: Reza Zadeh <rizlar@gmail.com>

Closes #2389 from rezazadeh/irmdocs and squashes the following commits:

7fa1313 [Reza Zadeh] Update svd docs
715da25 [Reza Zadeh] Updated computeSVD documentation IndexedRowMatrix
This commit is contained in:
Reza Zadeh 2014-09-15 17:41:15 -07:00 committed by Xiangrui Meng
parent 3b93128139
commit 983d6a9c48
1 changed files with 4 additions and 8 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/IndexedRowMatrix.scala
View file

@ -76,16 +76,12 @@ class IndexedRowMatrix(
}
/**
* Computes the singular value decomposition of this matrix.
* Computes the singular value decomposition of this IndexedRowMatrix.
* Denote this matrix by A (m x n), this will compute matrices U, S, V such that A = U * S * V'.
*
* There is no restriction on m, but we require `n^2` doubles to fit in memory.
* Further, n should be less than m.
* The decomposition is computed by first computing A'A = V S^2 V',
* computing svd locally on that (since n x n is small), from which we recover S and V.
* Then we compute U via easy matrix multiplication as U = A * (V * S^-1).
* Note that this approach requires `O(n^3)` time on the master node.
* The cost and implementation of this method is identical to that in
* [[org.apache.spark.mllib.linalg.distributed.RowMatrix]]
* With the addition of indices.
*
* At most k largest non-zero singular values and associated vectors are returned.
* If there are k such values, then the dimensions of the return will be: