spark-instrumented-optimizer/python/pyspark/pandas/tests
Xinrong Meng d1b24d8aba [SPARK-35338][PYTHON] Separate arithmetic operations into data type based structures
### What changes were proposed in this pull request?

The PR is proposed for **pandas APIs on Spark**, in order to separate arithmetic operations shown as below into data-type-based structures.
`__add__, __sub__, __mul__, __truediv__, __floordiv__, __pow__, __mod__,
__radd__, __rsub__, __rmul__, __rtruediv__, __rfloordiv__, __rpow__,__rmod__`

DataTypeOps and subclasses are introduced.

The existing behaviors of each arithmetic operation should be preserved.

### Why are the changes needed?

Currently, the same arithmetic operation of all data types is defined in one function, so it’s difficult to extend the behavior change based on the data types.

Introducing DataTypeOps would be the foundation for [pandas APIs on Spark: Separate basic operations into data type based structures.](https://docs.google.com/document/d/12MS6xK0hETYmrcl5b9pX5lgV4FmGVfpmcSKq--_oQlc/edit?usp=sharing).

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Tests are introduced under pyspark.pandas.tests.data_type_ops. One test file per DataTypeOps class.

Closes #32469 from xinrong-databricks/datatypeop_arith.

Authored-by: Xinrong Meng <xinrong.meng@databricks.com>
Signed-off-by: Takuya UESHIN <ueshin@databricks.com>
2021-05-19 15:05:32 -07:00
..
data_type_ops [SPARK-35338][PYTHON] Separate arithmetic operations into data type based structures 2021-05-19 15:05:32 -07:00
indexes [SPARK-35338][PYTHON] Separate arithmetic operations into data type based structures 2021-05-19 15:05:32 -07:00
plot [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
__init__.py [SPARK-34886][PYTHON] Port/integrate Koalas DataFrame unit test into PySpark 2021-04-09 15:48:13 +09:00
test_categorical.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_config.py [SPARK-35176][PYTHON] Standardize input validation error type 2021-05-03 15:34:24 +09:00
test_csv.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_dataframe.py [SPARK-35338][PYTHON] Separate arithmetic operations into data type based structures 2021-05-19 15:05:32 -07:00
test_dataframe_conversion.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_dataframe_spark_io.py [SPARK-35040][PYTHON] Remove Spark-version related codes from test codes 2021-04-22 18:01:07 -07:00
test_default_index.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_expanding.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_extension.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_frame_spark.py [SPARK-35040][PYTHON] Remove Spark-version related codes from test codes 2021-04-22 18:01:07 -07:00
test_groupby.py [SPARK-35176][PYTHON] Standardize input validation error type 2021-05-03 15:34:24 +09:00
test_indexing.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_indexops_spark.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_internal.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_namespace.py [SPARK-35176][PYTHON] Standardize input validation error type 2021-05-03 15:34:24 +09:00
test_numpy_compat.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_ops_on_diff_frames.py [SPARK-35176][PYTHON] Standardize input validation error type 2021-05-03 15:34:24 +09:00
test_ops_on_diff_frames_groupby.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_ops_on_diff_frames_groupby_expanding.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_ops_on_diff_frames_groupby_rolling.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_repr.py [SPARK-35040][PYTHON] Remove Spark-version related codes from test codes 2021-04-22 18:01:07 -07:00
test_reshape.py [SPARK-35040][PYTHON] Remove Spark-version related codes from test codes 2021-04-22 18:01:07 -07:00
test_rolling.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_series.py [SPARK-35176][PYTHON] Standardize input validation error type 2021-05-03 15:34:24 +09:00
test_series_conversion.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_series_datetime.py [SPARK-35338][PYTHON] Separate arithmetic operations into data type based structures 2021-05-19 15:05:32 -07:00
test_series_string.py [SPARK-35176][PYTHON] Standardize input validation error type 2021-05-03 15:34:24 +09:00
test_sql.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_stats.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00
test_typedef.py [SPARK-35071][PYTHON] Rename Koalas to pandas-on-Spark in main codes 2021-04-15 12:48:59 +09:00
test_utils.py [SPARK-35176][PYTHON] Standardize input validation error type 2021-05-03 15:34:24 +09:00
test_window.py [SPARK-34999][PYTHON] Consolidate PySpark testing utils 2021-04-22 13:07:35 -07:00