ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[MINOR][ML] Fix confusing error message in VectorAssembler

Browse source 
### What changes were proposed in this pull request?

When VectorAssembler encounters a NULL with handleInvalid="error", it throws an exception. This exception, though, has a typo making it confusing. Yet apparently, this same exception for NaN values is fine. Fixed it to look like the right one.

### Why are the changes needed?

Encountering this error with such message was very confusing! I hope to save time of fellow engineers by improving it.

### Does this PR introduce any user-facing change?

No.

### How was this patch tested?

It's just an error message...

Closes #27709 from Saluev/patch-1.

Authored-by: Tigran Saluev <tigran@saluev.com>
Signed-off-by: Sean Owen <srowen@gmail.com>
This commit is contained in:
Tigran Saluev 2020-02-27 11:05:53 -06:00 committed by Sean Owen
parent 2d2706cb86
commit 6f4a2e4c99
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala
View file

@ -288,7 +288,7 @@ object VectorAssembler extends DefaultParamsReadable[VectorAssembler] {
featureIndex += length featureIndex += length
} else { } else {
throw new SparkException( throw new SparkException(
s"""Encountered null while assembling a row with handleInvalid = "keep". Consider s"""Encountered null while assembling a row with handleInvalid = "error". Consider
|removing nulls from dataset or using handleInvalid = "keep" or "skip".""" |removing nulls from dataset or using handleInvalid = "keep" or "skip"."""
.stripMargin) .stripMargin)
} }