88542bc3d9
### What changes were proposed in this pull request?
PySpark UDF to convert MLlib vectors to dense arrays.
Example:
```
from pyspark.ml.functions import vector_to_array
df.select(vector_to_array(col("features"))
```
### Why are the changes needed?
If a PySpark user wants to convert MLlib sparse/dense vectors in a DataFrame into dense arrays, an efficient approach is to do that in JVM. However, it requires PySpark user to write Scala code and register it as a UDF. Often this is infeasible for a pure python project.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
UT.
Closes #26910 from WeichenXu123/vector_to_array.
Authored-by: WeichenXu <weichen.xu@databricks.com>
Signed-off-by: Xiangrui Meng <meng@databricks.com>
123 lines
2.2 KiB
ReStructuredText
123 lines
2.2 KiB
ReStructuredText
pyspark.ml package
|
|
==================
|
|
|
|
ML Pipeline APIs
|
|
----------------
|
|
|
|
.. automodule:: pyspark.ml
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.param module
|
|
-----------------------
|
|
|
|
.. automodule:: pyspark.ml.param
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.feature module
|
|
-------------------------
|
|
|
|
.. automodule:: pyspark.ml.feature
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.classification module
|
|
--------------------------------
|
|
|
|
.. automodule:: pyspark.ml.classification
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.clustering module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.clustering
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.functions module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.functions
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.linalg module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.linalg
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.recommendation module
|
|
--------------------------------
|
|
|
|
.. automodule:: pyspark.ml.recommendation
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.regression module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.regression
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.stat module
|
|
----------------------
|
|
|
|
.. automodule:: pyspark.ml.stat
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.tuning module
|
|
------------------------
|
|
|
|
.. automodule:: pyspark.ml.tuning
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.evaluation module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.evaluation
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.fpm module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.fpm
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.image module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.image
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.util module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.util
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|