spark-instrumented-optimizer/python/pyspark/ml
hyukjinkwon d9798c834f [SPARK-22313][PYTHON] Mark/print deprecation warnings as DeprecationWarning for deprecated APIs
## What changes were proposed in this pull request?

This PR proposes to mark the existing warnings as `DeprecationWarning` and print out warnings for deprecated functions.

This could be actually useful for Spark app developers. I use (old) PyCharm and this IDE can detect this specific `DeprecationWarning` in some cases:

**Before**

<img src="https://user-images.githubusercontent.com/6477701/31762664-df68d9f8-b4f6-11e7-8773-f0468f70a2cc.png" height="45" />

**After**

<img src="https://user-images.githubusercontent.com/6477701/31762662-de4d6868-b4f6-11e7-98dc-3c8446a0c28a.png" height="70" />

For console usage, `DeprecationWarning` is usually disabled (see https://docs.python.org/2/library/warnings.html#warning-categories and https://docs.python.org/3/library/warnings.html#warning-categories):

```
>>> import warnings
>>> filter(lambda f: f[2] == DeprecationWarning, warnings.filters)
[('ignore', <_sre.SRE_Pattern object at 0x10ba58c00>, <type 'exceptions.DeprecationWarning'>, <_sre.SRE_Pattern object at 0x10bb04138>, 0), ('ignore', None, <type 'exceptions.DeprecationWarning'>, None, 0)]
```

so, it won't actually mess up the terminal much unless it is intended.

If this is intendedly enabled, it'd should as below:

```
>>> import warnings
>>> warnings.simplefilter('always', DeprecationWarning)
>>>
>>> from pyspark.sql import functions
>>> functions.approxCountDistinct("a")
.../spark/python/pyspark/sql/functions.py:232: DeprecationWarning: Deprecated in 2.1, use approx_count_distinct instead.
  "Deprecated in 2.1, use approx_count_distinct instead.", DeprecationWarning)
...
```

These instances were found by:

```
cd python/pyspark
grep -r "Deprecated" .
grep -r "deprecated" .
grep -r "deprecate" .
```

## How was this patch tested?

Manually tested.

Author: hyukjinkwon <gurwls223@gmail.com>

Closes #19535 from HyukjinKwon/deprecated-warning.
2017-10-24 12:44:47 +09:00
..
linalg [SPARK-20214][ML] Make sure converted csc matrix has sorted indices 2017-04-05 17:46:44 -07:00
param [SPARK-21027][ML][PYTHON] Added tunable parallelism to one vs. rest in both Scala mllib and Pyspark 2017-09-12 10:02:27 -07:00
__init__.py [SPARK-21633][ML][PYTHON] UnaryTransformer in Python 2017-08-04 01:01:32 -07:00
base.py [SPARK-21633][ML][PYTHON] UnaryTransformer in Python 2017-08-04 01:01:32 -07:00
classification.py [SPARK-18608][ML][FOLLOWUP] Fix double caching for PySpark OneVsRest. 2017-09-14 14:09:44 +08:00
clustering.py [SPARK-10931][ML][PYSPARK] PySpark Models Copy Param Values from Estimator 2017-08-22 17:40:50 -07:00
common.py [SPARK-17679] [PYSPARK] remove unnecessary Py4J ListConverter patch 2016-10-03 14:12:03 -07:00
evaluation.py [SPARK-21981][PYTHON][ML] Added Python interface for ClusteringEvaluator 2017-09-22 13:12:33 +08:00
feature.py [SPARK-19866][ML][PYSPARK] Add local version of Word2Vec findSynonyms for spark.ml: Python API 2017-09-08 12:09:00 -07:00
fpm.py [SPARK-20768][PYSPARK][ML] Expose numPartitions (expert) param of PySpark FPGrowth. 2017-05-25 21:40:39 +08:00
pipeline.py [SPARK-17025][ML][PYTHON] Persistence for Pipelines with Python-only Stages 2017-08-11 23:57:08 -07:00
recommendation.py [SPARK-20679][ML] Support recommending for a subset of users/items in ALSModel 2017-10-09 10:42:33 +02:00
regression.py [SPARK-21523][ML] update breeze to 0.13.2 for an emergency bugfix in strong wolfe line search 2017-08-09 14:44:10 +08:00
stat.py [SPARK-20076][ML][PYSPARK] Add Python interface for ml.stats.Correlation 2017-04-07 11:00:10 +02:00
tests.py [SPARK-21854] Added LogisticRegressionTrainingSummary for MultinomialLogisticRegression in Python API 2017-09-14 13:53:28 +08:00
tuning.py [SPARK-21221][ML] CrossValidator and TrainValidationSplit Persist Nested Estimators such as OneVsRest 2017-07-17 10:07:32 -07:00
util.py [SPARK-22313][PYTHON] Mark/print deprecation warnings as DeprecationWarning for deprecated APIs 2017-10-24 12:44:47 +09:00
wrapper.py [SPARK-10931][ML][PYSPARK] PySpark Models Copy Param Values from Estimator 2017-08-22 17:40:50 -07:00