88542bc3d9
### What changes were proposed in this pull request? PySpark UDF to convert MLlib vectors to dense arrays. Example: ``` from pyspark.ml.functions import vector_to_array df.select(vector_to_array(col("features")) ``` ### Why are the changes needed? If a PySpark user wants to convert MLlib sparse/dense vectors in a DataFrame into dense arrays, an efficient approach is to do that in JVM. However, it requires PySpark user to write Scala code and register it as a UDF. Often this is infeasible for a pure python project. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? UT. Closes #26910 from WeichenXu123/vector_to_array. Authored-by: WeichenXu <weichen.xu@databricks.com> Signed-off-by: Xiangrui Meng <meng@databricks.com>
123 lines
2.2 KiB
ReStructuredText
123 lines
2.2 KiB
ReStructuredText
pyspark.ml package
|
|
==================
|
|
|
|
ML Pipeline APIs
|
|
----------------
|
|
|
|
.. automodule:: pyspark.ml
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.param module
|
|
-----------------------
|
|
|
|
.. automodule:: pyspark.ml.param
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.feature module
|
|
-------------------------
|
|
|
|
.. automodule:: pyspark.ml.feature
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.classification module
|
|
--------------------------------
|
|
|
|
.. automodule:: pyspark.ml.classification
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.clustering module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.clustering
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.functions module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.functions
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.linalg module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.linalg
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.recommendation module
|
|
--------------------------------
|
|
|
|
.. automodule:: pyspark.ml.recommendation
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.regression module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.regression
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.stat module
|
|
----------------------
|
|
|
|
.. automodule:: pyspark.ml.stat
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.tuning module
|
|
------------------------
|
|
|
|
.. automodule:: pyspark.ml.tuning
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.evaluation module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.evaluation
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.fpm module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.fpm
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.image module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.image
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|
|
|
|
pyspark.ml.util module
|
|
----------------------------
|
|
|
|
.. automodule:: pyspark.ml.util
|
|
:members:
|
|
:undoc-members:
|
|
:inherited-members:
|