921abc51cf
### What changes were proposed in this pull request? This PR proposes to restructure API files according to the layout, see https://github.com/apache/spark/pull/32799. Now the pandas APIs on Spark are under a separate directory which is same level as other modules such as Spark SQL. ```bash tree reference ``` **Before:** ``` reference ├── index.rst ├── ps_extensions.rst ├── ps_frame.rst ├── ps_general_functions.rst ├── ps_groupby.rst ├── ps_indexing.rst ├── ps_io.rst ├── ps_ml.rst ├── ps_series.rst ├── ps_window.rst ├── pyspark.ml.rst ├── pyspark.mllib.rst ├── pyspark.pandas.rst ├── pyspark.resource.rst ├── pyspark.rst ├── pyspark.sql.rst ├── pyspark.ss.rst └── pyspark.streaming.rst ``` **After:** ``` reference ├── index.rst ├── pyspark.ml.rst ├── pyspark.mllib.rst ├── pyspark.pandas │ ├── extensions.rst │ ├── frame.rst │ ├── general_functions.rst │ ├── groupby.rst │ ├── index.rst │ ├── indexing.rst │ ├── io.rst │ ├── ml.rst │ ├── series.rst │ └── window.rst ├── pyspark.resource.rst ├── pyspark.rst ├── pyspark.sql.rst ├── pyspark.ss.rst └── pyspark.streaming.rst ``` ### Why are the changes needed? To make the directory structure easier to follow. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually built and tested the docs. Closes #32812 from HyukjinKwon/SPARK-35646-followup. Authored-by: Hyukjin Kwon <gurwls223@apache.org> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
29 lines
723 B
ReStructuredText
29 lines
723 B
ReStructuredText
.. _api.ml:
|
|
|
|
==========================
|
|
Machine Learning utilities
|
|
==========================
|
|
.. currentmodule:: pyspark.pandas.mlflow
|
|
|
|
MLflow
|
|
------
|
|
|
|
Arbitrary MLflow models can be used with Koalas Dataframes,
|
|
provided they implement the 'pyfunc' flavor. This is the case
|
|
for most frameworks supported by MLflow (scikit-learn, pytorch,
|
|
tensorflow, ...). See comprehensive examples in
|
|
:func:`load_model` for more information.
|
|
|
|
.. note::
|
|
The MLflow package must be installed in order to use this module.
|
|
If MLflow is not installed in your environment already, you
|
|
can install it with the following command:
|
|
|
|
**pip install koalas[mlflow]**
|
|
|
|
.. autosummary::
|
|
:toctree: api/
|
|
|
|
PythonModelWrapper
|
|
load_model
|