spark-instrumented-optimizer/python/pyspark/pandas
Hyukjin Kwon 7eb74482a7 [SPARK-35510][PYTHON] Fix and reenable test_stats_on_non_numeric_columns_should_be_discarded_if_numeric_only_is_true
### What changes were proposed in this pull request?

This PR proposes to fix and reenable `test_stats_on_non_numeric_columns_should_be_discarded_if_numeric_only_is_true` that was disabled when we upgrade Python 3.9 in CI at https://github.com/apache/spark/pull/32657.

Seems like this is because of the latest NumPy's behaviour change, see also `https://github.com/numpy/numpy/pull/16273#discussion_r641264085`.

pandas inherits this behaviour but it doesn't make sense when `numeric_only` is set to `True` in pandas. I will track and follow the status of the issue between pandas and NumPy.

For the time being, I propose to exclude boolean case alone in percentile/quartile test case

### Why are the changes needed?

To keep the test coverage.

### Does this PR introduce _any_ user-facing change?

No, test-only.

### How was this patch tested?

I roughly locally tested. But it should pass in CI.

Closes #32690 from HyukjinKwon/SPARK-35510.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
2021-05-28 17:35:01 +09:00
..
data_type_ops [SPARK-35522][PYTHON] Introduce BinaryOps for BinaryType 2021-05-26 14:30:24 -07:00
indexes [SPARK-35505][PYTHON] Remove APIs which have been deprecated in Koalas 2021-05-25 11:16:27 -07:00
missing [SPARK-35071][PYTHON] Rename Koalas to pandas-on-Spark in main codes 2021-04-15 12:48:59 +09:00
plot [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
spark [SPARK-35467][SPARK-35468][SPARK-35477][PYTHON] Fix disallow_untyped_defs mypy checks 2021-05-24 09:31:00 +09:00
tests [SPARK-35510][PYTHON] Fix and reenable test_stats_on_non_numeric_columns_should_be_discarded_if_numeric_only_is_true 2021-05-28 17:35:01 +09:00
typedef [SPARK-35467][SPARK-35468][SPARK-35477][PYTHON] Fix disallow_untyped_defs mypy checks 2021-05-24 09:31:00 +09:00
usage_logging [SPARK-35071][PYTHON] Rename Koalas to pandas-on-Spark in main codes 2021-04-15 12:48:59 +09:00
__init__.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
accessors.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
base.py [SPARK-35505][PYTHON] Remove APIs which have been deprecated in Koalas 2021-05-25 11:16:27 -07:00
categorical.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
config.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
datetimes.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
exceptions.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
extensions.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
frame.py [SPARK-35505][PYTHON] Remove APIs which have been deprecated in Koalas 2021-05-25 11:16:27 -07:00
generic.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
groupby.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00
indexing.py [SPARK-35537][PYTHON] Introduce a util function spark_column_equals 2021-05-27 12:14:43 +09:00
internal.py [SPARK-35537][PYTHON] Introduce a util function spark_column_equals 2021-05-27 12:14:43 +09:00
ml.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
mlflow.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
namespace.py [SPARK-35467][SPARK-35468][SPARK-35477][PYTHON] Fix disallow_untyped_defs mypy checks 2021-05-24 09:31:00 +09:00
numpy_compat.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
series.py [SPARK-35505][PYTHON] Remove APIs which have been deprecated in Koalas 2021-05-25 11:16:27 -07:00
sql_processor.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
strings.py [SPARK-35465][PYTHON] Set up the mypy configuration to enable disallow_untyped_defs check for pandas APIs on Spark module 2021-05-21 11:03:35 -07:00
utils.py [SPARK-35537][PYTHON] Introduce a util function spark_column_equals 2021-05-27 12:14:43 +09:00
version.py [SPARK-34890][PYTHON] Port/integrate Koalas main codes into PySpark 2021-04-06 12:42:39 +09:00
window.py [SPARK-35364][PYTHON] Renaming the existing Koalas related codes 2021-05-20 15:08:30 -07:00