spark-instrumented-optimizer/python/pyspark/pandas/tests/indexes
Takuya UESHIN bcc595c112 [SPARK-36310][PYTHON] Fix IndexOpsMixin.hasnans to use isnull().any()
### What changes were proposed in this pull request?

Fix `IndexOpsMixin.hasnans` to use `IndexOpsMixin.isnull().any()`.

### Why are the changes needed?

`IndexOpsMixin.hasnans` has a potential issue to cause `a window function inside an aggregate function` error.
Also it returns a wrong value when the `Series`/`Index` is empty.

```py
>>> ps.Series([]).hasnans
None
```

whereas:

```py
>>> pd.Series([]).hasnans
False
```

`IndexOpsMixin.any()` is safe for both cases.

### Does this PR introduce _any_ user-facing change?

`IndexOpsMixin.hasnans` will return `False` when empty.

### How was this patch tested?

Added some tests.

Closes #33547 from ueshin/issues/SPARK-36310/hasnan.

Authored-by: Takuya UESHIN <ueshin@databricks.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
2021-07-28 09:21:12 +09:00
..
__init__.py [SPARK-35032][PYTHON] Port Koalas Index unit tests into PySpark 2021-04-16 08:53:30 +09:00
test_base.py [SPARK-36310][PYTHON] Fix IndexOpsMixin.hasnans to use isnull().any() 2021-07-28 09:21:12 +09:00
test_category.py [SPARK-36260][PYTHON] Add set_categories to CategoricalAccessor and CategoricalIndex 2021-07-26 17:12:33 -07:00
test_datetime.py [SPARK-35340][PYTHON] Standardize TypeError messages for unsupported basic operations 2021-07-08 12:27:48 -07:00