ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/python/pyspark/ml
History
hyukjinkwon f7435bec6a [SPARK-24044][PYTHON] Explicitly print out skipped tests from unittest module 
## What changes were proposed in this pull request?

This PR proposes to remove duplicated dependency checking logics and also print out skipped tests from unittests.

For example, as below:

```
Skipped tests in pyspark.sql.tests with pypy:
    test_createDataFrame_column_name_encoding (pyspark.sql.tests.ArrowTests) ... skipped 'Pandas >= 0.19.2 must be installed; however, it was not found.'
    test_createDataFrame_does_not_modify_input (pyspark.sql.tests.ArrowTests) ... skipped 'Pandas >= 0.19.2 must be installed; however, it was not found.'
...

Skipped tests in pyspark.sql.tests with python3:
    test_createDataFrame_column_name_encoding (pyspark.sql.tests.ArrowTests) ... skipped 'PyArrow >= 0.8.0 must be installed; however, it was not found.'
    test_createDataFrame_does_not_modify_input (pyspark.sql.tests.ArrowTests) ... skipped 'PyArrow >= 0.8.0 must be installed; however, it was not found.'
...
```

Currently, it's not printed out in the console. I think we should better print out skipped tests in the console.

## How was this patch tested?

Manually tested. Also, fortunately, Jenkins has good environment to test the skipped output.

Author: hyukjinkwon <gurwls223@apache.org>

Closes #21107 from HyukjinKwon/skipped-tests-print.
2018-04-26 15:11:42 -07:00
..
linalg [SPARK-23522][PYTHON] always use sys.exit over builtin exit 2018-03-08 20:38:34 +09:00
param [SPARK-21088][ML] CrossValidator, TrainValidationSplit support collect all models when fitting: Python API 2018-04-16 11:31:24 -05:00
__init__.py [SPARK-21633][ML][PYTHON] UnaryTransformer in Python 2017-08-04 01:01:32 -07:00
base.py [SPARK-22922][ML][PYSPARK] Pyspark portion of the fit-multiple API 2017-12-29 16:31:25 -08:00
classification.py typo rawPredicition changed to rawPrediction 2018-04-11 15:52:13 -07:00
clustering.py [SPARK-23522][PYTHON] always use sys.exit over builtin exit 2018-03-08 20:38:34 +09:00
common.py [SPARK-17679] [PYSPARK] remove unnecessary Py4J ListConverter patch 2016-10-03 14:12:03 -07:00
evaluation.py [SPARK-23522][PYTHON] always use sys.exit over builtin exit 2018-03-08 20:38:34 +09:00
feature.py [SPARK-23871][ML][PYTHON] add python api for VectorAssembler handleInvalid 2018-04-10 15:41:45 -07:00
fpm.py [SPARK-23163][DOC][PYTHON] Sync ML Python API with Scala 2018-01-25 01:48:11 -08:00
image.py [SPARK-23522][PYTHON] always use sys.exit over builtin exit 2018-03-08 20:38:34 +09:00
pipeline.py [SPARK-17025][ML][PYTHON] Persistence for Pipelines with Python-only Stages 2017-08-11 23:57:08 -07:00
recommendation.py [SPARK-23522][PYTHON] always use sys.exit over builtin exit 2018-03-08 20:38:34 +09:00
regression.py [SPARK-23162][PYSPARK][ML] Add r2adj into Python API in LinearRegressionSummary 2018-03-26 15:45:27 -07:00
stat.py [SPARK-21741][ML][PYSPARK] Python API for DataFrame-based multivariate summarizer 2018-04-17 10:11:08 -07:00
tests.py [SPARK-24044][PYTHON] Explicitly print out skipped tests from unittest module 2018-04-26 15:11:42 -07:00
tuning.py [SPARK-21088][ML] CrossValidator, TrainValidationSplit support collect all models when fitting: Python API 2018-04-16 11:31:24 -05:00
util.py [SPARK-21088][ML] CrossValidator, TrainValidationSplit support collect all models when fitting: Python API 2018-04-16 11:31:24 -05:00
wrapper.py [SPARK-21685][PYTHON][ML] PySpark Params isSet state should not change after transform 2018-03-23 11:42:40 -07:00