ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-33894][SQL] Change visibility of private case classes in mllib to avoid runtime compilation errors with Scala 2.13

Browse source 
### What changes were proposed in this pull request?
Change visibility modifier of two case classes defined inside objects in mllib from private to private[OuterClass]

### Why are the changes needed?
Without this change when running tests for Scala 2.13 you get runtime code generation errors. These errors look like this:
```
[info] Cause: java.util.concurrent.ExecutionException: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 73, Column 65: failed to compile: org.codehaus.commons.compiler.CompileException: File 'generated.java', Line 73, Column 65: No applicable constructor/method found for zero actual parameters; candidates are: "public java.lang.String org.apache.spark.ml.feature.Word2VecModel$Data.word()"
```

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Existing tests now pass for Scala 2.13

Closes #31018 from koertkuipers/feat-visibility-scala213.

Authored-by: Koert Kuipers <koert@tresata.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
This commit is contained in:
Koert Kuipers 2021-01-04 15:40:32 -08:00 committed by Dongjoon Hyun
parent 84c1f43669
commit 9b4173fa95
2 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
mllib/src/main/scala/org/apache/spark/ml/feature/Word2Vec.scala
View file

@ -344,7 +344,7 @@ class Word2VecModel private[ml] (
@Since("1.6.0")
object Word2VecModel extends MLReadable[Word2VecModel] {
private case class Data(word: String, vector: Array[Float])
private[Word2VecModel] case class Data(word: String, vector: Array[Float])
private[Word2VecModel]
class Word2VecModelWriter(instance: Word2VecModel) extends MLWriter {

4
mllib/src/main/scala/org/apache/spark/mllib/clustering/KMeansModel.scala
View file

@ -145,9 +145,9 @@ object KMeansModel extends Loader[KMeansModel] {
}
}
private case class Cluster(id: Int, point: Vector)
private[KMeansModel] case class Cluster(id: Int, point: Vector)
private object Cluster {
private[KMeansModel] object Cluster {
def apply(r: Row): Cluster = {
Cluster(r.getInt(0), r.getAs[Vector](1))
}