4a549f2de2
### What changes were proposed in this pull request? There are several pr to fix compilation warnings related to `procedure syntax` like SPARK-29291, SPARK-33352 and SPARK-35526, in order to prevent the recurrence of similar problems, this pr add a compile arg to convert `procedure syntax` related compilation warnings to compilation errors in Scala 2.13. ### Why are the changes needed? Prevent the recurrence of compilation warnings related to `procedure syntax is deprecated` ### Does this PR introduce _any_ user-facing change? `procedure syntax` is no longer allowed in Spark code with Scala 2.13, for constructors methods definition should be `this(...) = { }` not `this(...) { }`, for without `return type` methods definition should be `def methodName(...): Unit = {}` not `def methodName(...) {}`. ### How was this patch tested? - Pass the GitHub Action Scala 2.13 job - Manual test: Do some code change like: ``` Index: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala =================================================================== -67,7 +67,7 private[spark] class HeartbeatReceiver(sc: SparkContext, clock: Clock) extends SparkListener with ThreadSafeRpcEndpoint with Logging { - def this(sc: SparkContext) = { + def this(sc: SparkContext) { this(sc, new SystemClock) } Index: core/src/main/scala/org/apache/spark/MapOutputTracker.scala =================================================================== -720,7 +720,7 } } - def registerMergeResult(shuffleId: Int, reduceId: Int, status: MergeStatus): Unit = { + def registerMergeResult(shuffleId: Int, reduceId: Int, status: MergeStatus) { shuffleStatuses(shuffleId).addMergeResult(reduceId, status) } ``` **sbt with Scala 2.13 profile compile failed as follows:*** ``` [error] /home/runner/work/spark/spark/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala:70:29: procedure syntax is deprecated for constructors: add `=`, as in method definition [error] def this(sc: SparkContext) { [error] ^ [error] /home/runner/work/spark/spark/core/src/main/scala/org/apache/spark/MapOutputTracker.scala:723:79: procedure syntax is deprecated: instead, add `: Unit =` to explicitly declare `registerMergeResult`'s return type [error] def registerMergeResult(shuffleId: Int, reduceId: Int, status: MergeStatus) { [error] ^ [error] two errors found [error] (core / Compile / compileIncremental) Compilation failed [error] Total time: 136 s (02:16), completed May 31, 2021 10:06:50 AM Error: Process completed with exit code 1. ``` **maven with Scala 2.13 profile compile failed as follows:** ``` [ERROR] [Error] /Users/yangjie01/SourceCode/git/spark-mine/core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala:70: procedure syntax is deprecated for constructors: add `=`, as in method definition [ERROR] [Error] /Users/yangjie01/SourceCode/git/spark-mine/core/src/main/scala/org/apache/spark/MapOutputTracker.scala:723: procedure syntax is deprecated: instead, add `: Unit =` to explicitly declare `registerMergeResult`'s return type [ERROR] two errors found ``` Closes #32710 from LuciferYang/SPARK-35574. Authored-by: yangjie01 <yangjie01@baidu.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org> |
||
---|---|---|
.. | ||
build.properties | ||
MimaBuild.scala | ||
MimaExcludes.scala | ||
plugins.sbt | ||
SparkBuild.scala |