From 79a6e00b7621bb3ebd99ff692452d292e11fb377 Mon Sep 17 00:00:00 2001 From: Dongjoon Hyun Date: Thu, 1 Jul 2021 22:24:48 -0700 Subject: [PATCH] [SPARK-35825][INFRA][FOLLOWUP] Increase it in build/mvn script ### What changes were proposed in this pull request? This is a follow up of https://github.com/apache/spark/pull/32961. This PR additionally sets the stack size in `build/mvn`. ### Why are the changes needed? We are still hitting `StackOverflowError` in Jenkins. - https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-3.2/3064/console ``` [INFO] compiling 166 Scala sources and 19 Java sources to /home/jenkins/workspace/spark-master-test-maven-hadoop-3.2/sql/catalyst/target/scala-2.12/classes ... [ERROR] ## Exception when compiling 480 sources to /home/jenkins/workspace/spark-master-test-maven-hadoop-3.2/sql/catalyst/target/scala-2.12/classes java.lang.StackOverflowError ``` This PR increases the JVM of `mvn` instead of the plugin. ``` $ MAVEN_OPTS="-XX:+PrintFlagsFinal" build/mvn clean | grep 'intx ThreadStackSize' intx ThreadStackSize = 2048 {pd product} $ MAVEN_OPTS="-Xss128m -XX:+PrintFlagsFinal" build/mvn clean | grep 'intx ThreadStackSize' intx ThreadStackSize := 131072 {pd product} ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? N/A Closes #33180 from dongjoon-hyun/SPARK-35825. Authored-by: Dongjoon Hyun Signed-off-by: Dongjoon Hyun --- build/mvn | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/build/mvn b/build/mvn index 60c6164010..bf12347c0f 100755 --- a/build/mvn +++ b/build/mvn @@ -29,7 +29,7 @@ _DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" # Preserve the calling directory _CALLING_DIR="$(pwd)" # Options used during compilation -_COMPILE_JVM_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g" +_COMPILE_JVM_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=1g -Xss128m" # Installs any application tarball given a URL, the expected tarball name, # and, optionally, a checkable binary path to determine if the binary has