31a16fbb40
### What changes were proposed in this pull request? This PR proposes migration of [`pyspark-stubs`](https://github.com/zero323/pyspark-stubs) into Spark codebase. ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? Yes. This PR adds type annotations directly to Spark source. This can impact interaction with development tools for users, which haven't used `pyspark-stubs`. ### How was this patch tested? - [x] MyPy tests of the PySpark source ``` mypy --no-incremental --config python/mypy.ini python/pyspark ``` - [x] MyPy tests of Spark examples ``` MYPYPATH=python/ mypy --no-incremental --config python/mypy.ini examples/src/main/python/ml examples/src/main/python/sql examples/src/main/python/sql/streaming ``` - [x] Existing Flake8 linter - [x] Existing unit tests Tested against: - `mypy==0.790+dev.e959952d9001e9713d329a2f9b196705b028f894` - `mypy==0.782` Closes #29591 from zero323/SPARK-32681. Authored-by: zero323 <mszymkiewicz@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org> |
||
---|---|---|
.. | ||
linalg | ||
stat | ||
tests | ||
__init__.py | ||
__init__.pyi | ||
_typing.pyi | ||
classification.py | ||
classification.pyi | ||
clustering.py | ||
clustering.pyi | ||
common.py | ||
common.pyi | ||
evaluation.py | ||
evaluation.pyi | ||
feature.py | ||
feature.pyi | ||
fpm.py | ||
fpm.pyi | ||
random.py | ||
random.pyi | ||
recommendation.py | ||
recommendation.pyi | ||
regression.py | ||
regression.pyi | ||
tree.py | ||
tree.pyi | ||
util.py | ||
util.pyi |