ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-25758][ML] Deprecate computeCost in BisectingKMeans

Browse source 
## What changes were proposed in this pull request?

The PR proposes to deprecate the `computeCost` method on `BisectingKMeans` in favor of the adoption of `ClusteringEvaluator` in order to evaluate the clustering.

## How was this patch tested?

NA

Closes #22869 from mgaido91/SPARK-25758_3.0.

Authored-by: Marco Gaido <marcogaido91@gmail.com>
Signed-off-by: DB Tsai <d_tsai@apple.com>
This commit is contained in:
Marco Gaido 2018-11-05 22:13:20 +00:00 committed by DB Tsai
parent fc65b4af00
commit fc10c898f4
No known key found for this signature in database
GPG key ID: E6FD79DA81FE14FD
2 changed files with 14 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
mllib/src/main/scala/org/apache/spark/ml/clustering/BisectingKMeans.scala
View file

@ -125,8 +125,15 @@ class BisectingKMeansModel private[ml] (
/**
* Computes the sum of squared distances between the input points and their corresponding cluster
* centers.
*
* @deprecated This method is deprecated and will be removed in future versions. Use
* ClusteringEvaluator instead. You can also get the cost on the training dataset in
* the summary.
*/
@Since("2.0.0")
@deprecated("This method is deprecated and will be removed in future versions. Use " +
"ClusteringEvaluator instead. You can also get the cost on the training dataset in the " +
"summary.", "3.0.0")
def computeCost(dataset: Dataset[_]): Double = {
SchemaUtils.validateVectorCompatibleColumn(dataset.schema, getFeaturesCol)
val data = DatasetUtils.columnToOldVector(dataset, getFeaturesCol)

7
python/pyspark/ml/clustering.py
View file

@ -540,7 +540,14 @@ class BisectingKMeansModel(JavaModel, JavaMLWritable, JavaMLReadable):
"""
Computes the sum of squared distances between the input points
and their corresponding cluster centers.
..note:: Deprecated in 3.0.0. It will be removed in future versions. Use
ClusteringEvaluator instead. You can also get the cost on the training dataset in the
summary.
"""
warnings.warn("Deprecated in 3.0.0. It will be removed in future versions. Use "
"ClusteringEvaluator instead. You can also get the cost on the training "
"dataset in the summary.", DeprecationWarning)
return self._call_java("computeCost", dataset)
@property