ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-5905] [MLLIB] Note requirements for certain RowMatrix methods in docs

Browse source 
Note methods that fail for cols > 65535; note that SVD does not require n >= m
CC mengxr

Author: Sean Owen <sowen@cloudera.com>

Closes #8839 from srowen/SPARK-5905.
This commit is contained in:
Sean Owen 2015-09-20 16:05:12 -07:00 committed by Xiangrui Meng
parent 2117eea71e
commit 1aa9e50256
1 changed files with 8 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
mllib/src/main/scala/org/apache/spark/mllib/linalg/distributed/RowMatrix.scala
View file

@ -109,7 +109,8 @@ class RowMatrix @Since("1.0.0") (
}
/**
* Computes the Gramian matrix `A^T A`.
* Computes the Gramian matrix `A^T A`. Note that this cannot be computed on matrices with
* more than 65535 columns.
*/
@Since("1.0.0")
def computeGramianMatrix(): Matrix = {
@ -150,7 +151,8 @@ class RowMatrix @Since("1.0.0") (
* - s is a Vector of size k, holding the singular values in descending order,
* - V is a Matrix of size n x k that satisfies V' * V = eye(k).
*
* We assume n is smaller than m. The singular values and the right singular vectors are derived
* We assume n is smaller than m, though this is not strictly required.
* The singular values and the right singular vectors are derived
* from the eigenvalues and the eigenvectors of the Gramian matrix A' * A. U, the matrix
* storing the right singular vectors, is computed via matrix multiplication as
* U = A * (V * S^-1^), if requested by user. The actual method to use is determined
@ -320,7 +322,8 @@ class RowMatrix @Since("1.0.0") (
}
/**
* Computes the covariance matrix, treating each row as an observation.
* Computes the covariance matrix, treating each row as an observation. Note that this cannot
* be computed on matrices with more than 65535 columns.
* @return a local dense matrix of size n x n
*/
@Since("1.0.0")
@ -374,6 +377,8 @@ class RowMatrix @Since("1.0.0") (
* The row data do not need to be "centered" first; it is not necessary for
* the mean of each column to be 0.
*
* Note that this cannot be computed on matrices with more than 65535 columns.
*
* @param k number of top principal components.
* @return a matrix of size n-by-k, whose columns are principal components
*/