ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/python/pyspark/pandas
History
Takuya UESHIN 2616d5cc1d [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 
### What changes were proposed in this pull request?

Sets up the `mypy` configuration to enable `disallow_untyped_defs` check for pandas APIs on Spark module.

### Why are the changes needed?

Currently many functions in the main codes in pandas APIs on Spark module are still missing type annotations and disabled `mypy` check `disallow_untyped_defs`.

We should add more type annotations and enable the mypy check.

### Does this PR introduce _any_ user-facing change?

Yes.
This PR adds more type annotations in pandas APIs on Spark module, which can impact interaction with development tools for users.

### How was this patch tested?

The mypy check with a new configuration and existing tests should pass.

Closes #32614 from ueshin/issues/SPARK-35465/disallow_untyped_defs.

Authored-by: Takuya UESHIN <ueshin@databricks.com>
Signed-off-by: Takuya UESHIN <ueshin@databricks.com>
2021-05-21 11:03:35 -07:00
..
data_type_ops [SPARK-35338][PYTHON] Separate arithmetic operations into data type based structures 2021-05-19 19:47:00 -07:00
indexes [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
missing [SPARK-35071][PYTHON] Rename Koalas to pandas-on-Spark in main codes 2021-04-15 12:48:59 +09:00
plot [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
spark [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
tests [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
typedef [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
usage_logging [SPARK-35071][PYTHON] Rename Koalas to pandas-on-Spark in main codes 2021-04-15 12:48:59 +09:00
__init__.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
accessors.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
base.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
categorical.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
config.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
datetimes.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
exceptions.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
extensions.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
frame.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
generic.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
groupby.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
indexing.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
internal.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
ml.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
mlflow.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
namespace.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
numpy_compat.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
series.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
sql_processor.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
strings.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
utils.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
version.py [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark 2021-04-06 12:42:39 +09:00
window.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00