From 3af1651e50be3bc2e441be8827441f87d34d99cc Mon Sep 17 00:00:00 2001 From: HyukjinKwon Date: Fri, 30 Oct 2020 17:53:30 +0900 Subject: [PATCH] [SPARK-33297][BUILD] Switch to use flat class loader strategy in SBT ### What changes were proposed in this pull request? This PR proposes to switch the class loader strategy from `ScalaLibrary` to `Flat` (see https://www.scala-sbt.org/1.x/docs/In-Process-Classloaders.html): https://github.com/apache/spark/runs/1314691686 ``` Error: java.util.MissingResourceException: Can't find bundle for base name org.scalactic.ScalacticBundle, locale en Error: at java.util.ResourceBundle.throwMissingResourceException(ResourceBundle.java:1581) Error: at java.util.ResourceBundle.getBundleImpl(ResourceBundle.java:1396) Error: at java.util.ResourceBundle.getBundle(ResourceBundle.java:782) Error: at org.scalactic.Resources$.resourceBundle$lzycompute(Resources.scala:8) Error: at org.scalactic.Resources$.resourceBundle(Resources.scala:8) Error: at org.scalactic.Resources$.pleaseDefineScalacticFillFilePathnameEnvVar(Resources.scala:256) Error: at org.scalactic.source.PositionMacro$PositionMacroImpl.apply(PositionMacro.scala:65) Error: at org.scalactic.source.PositionMacro$.genPosition(PositionMacro.scala:85) Error: at sun.reflect.GeneratedMethodAccessor34.invoke(Unknown Source) Error: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Error: at java.lang.reflect.Method.invoke(Method.java:498) ``` See also https://github.com/sbt/sbt/issues/5736 ### Why are the changes needed? To make the build unflaky. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? GitHub Actions build in this test. Closes #30198 from HyukjinKwon/SPARK-33297. Authored-by: HyukjinKwon Signed-off-by: HyukjinKwon --- project/SparkBuild.scala | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index 5f2ef480f8..55c87fcb3a 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -322,7 +322,11 @@ object SparkBuild extends PomBuild { // disable Mima check for all modules, // to be enabled in specific ones that have previous artifacts - MimaKeys.mimaFailOnNoPrevious := false + MimaKeys.mimaFailOnNoPrevious := false, + + // To prevent intermittent compliation failures, see also SPARK-33297 + // Apparently we can remove this when we use JDK 11. + Test / classLoaderLayeringStrategy := ClassLoaderLayeringStrategy.Flat ) def enable(settings: Seq[Setting[_]])(projectRef: ProjectRef) = {