ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-2091][MLLIB] use numpy.dot instead of ndarray.dot

Browse source 
`ndarray.dot` is not available in numpy 1.4. This PR makes pyspark/mllib compatible with numpy 1.4.

Author: Xiangrui Meng <meng@databricks.com>

Closes #1035 from mengxr/numpy-1.4 and squashes the following commits:

7ad2f0c [Xiangrui Meng] use numpy.dot instead of ndarray.dot
This commit is contained in:
Xiangrui Meng 2014-06-11 00:22:40 -07:00 committed by Reynold Xin
parent 0266a0c8a7
commit 0f1dc3a73d
1 changed files with 5 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
python/pyspark/mllib/_common.py
View file

@ -454,7 +454,7 @@ def _squared_distance(v1, v2):
v2 = _convert_vector(v2)
if type(v1) == ndarray and type(v2) == ndarray:
diff = v1 - v2
return diff.dot(diff)
return numpy.dot(diff, diff)
elif type(v1) == ndarray:
return v2.squared_distance(v1)
else:
@ -469,10 +469,12 @@ def _dot(vec, target):
calling numpy.dot of the two vectors, but for SciPy ones, we
have to transpose them because they're column vectors.
"""
if type(vec) == ndarray or type(vec) == SparseVector:
if type(vec) == ndarray:
return numpy.dot(vec, target)
elif type(vec) == SparseVector:
return vec.dot(target)
elif type(vec) == list:
return _convert_vector(vec).dot(target)
return numpy.dot(_convert_vector(vec), target)
else:
return vec.transpose().dot(target)[0]