ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-30121][BUILD] Fix memory usage in sbt build script

Browse source 
### What changes were proposed in this pull request?
1. the default memory setting is missing in usage instructions
```
build/sbt -h
```
before
```
-mem    <integer>  set memory options (default: , which is -Xms2048m -Xmx2048m -XX:ReservedCodeCacheSize=256m)
```
after
```
-mem    <integer>  set memory options (default: 2048, which is -Xms2048m -Xmx2048m -XX:ReservedCodeCacheSize=256m)
```
2. the Perm space is not needed anymore, since java7 is removed.

the changes in this pr are based on the main sbt script of the newest stable version 1.3.4.

### Why are the changes needed?

bug fix

### Does this PR introduce any user-facing change?

no

### How was this patch tested?

manually

Closes #26757 from yaooqinn/SPARK-30121.

Authored-by: Kent Yao <yaooqinn@hotmail.com>
Signed-off-by: Sean Owen <sean.owen@databricks.com>
This commit is contained in:
Kent Yao 2019-12-05 11:50:55 -06:00 committed by Sean Owen
parent b9cae37750
commit 35bab33984
2 changed files with 6 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
build/sbt
View file

@ -66,7 +66,7 @@ Usage: $script_name [options]
-sbt-dir <path> path to global settings/plugins directory (default: ~/.sbt) -sbt-dir <path> path to global settings/plugins directory (default: ~/.sbt)
-sbt-boot <path> path to shared boot directory (default: ~/.sbt/boot in 0.11 series) -sbt-boot <path> path to shared boot directory (default: ~/.sbt/boot in 0.11 series)
-ivy <path> path to local Ivy repository (default: ~/.ivy2) -ivy <path> path to local Ivy repository (default: ~/.ivy2)
-mem <integer> set memory options (default: $sbt_mem, which is $(get_mem_opts $sbt_mem)) -mem <integer> set memory options (default: $sbt_default_mem, which is $(get_mem_opts $sbt_default_mem))
-no-share use all local caches; no sharing -no-share use all local caches; no sharing
-no-global uses global caches, but does not use global ~/.sbt directory. -no-global uses global caches, but does not use global ~/.sbt directory.
-jvm-debug <port> Turn on JVM debugging, open at the given port. -jvm-debug <port> Turn on JVM debugging, open at the given port.

10
build/sbt-launch-lib.bash
View file

@ -17,6 +17,7 @@ declare -a java_args
declare -a scalac_args declare -a scalac_args
declare -a sbt_commands declare -a sbt_commands
declare -a maven_profiles declare -a maven_profiles
declare sbt_default_mem=2048
if test -x "$JAVA_HOME/bin/java"; then if test -x "$JAVA_HOME/bin/java"; then
echo -e "Using $JAVA_HOME as default JAVA_HOME." echo -e "Using $JAVA_HOME as default JAVA_HOME."
@ -111,11 +112,10 @@ addDebugger () {
# a ham-fisted attempt to move some memory settings in concert # a ham-fisted attempt to move some memory settings in concert
# so they need not be dicked around with individually. # so they need not be dicked around with individually.
get_mem_opts () { get_mem_opts () {
local mem=${1:-2048} local mem=${1:-$sbt_default_mem}
local perm=$(( $mem / 4 )) local codecache=$(( $mem / 8 ))
(( $perm > 256 )) || perm=256 (( $codecache > 128 )) || codecache=128
(( $perm < 4096 )) || perm=4096 (( $codecache < 2048 )) || codecache=2048
local codecache=$(( $perm / 2 ))
echo "-Xms${mem}m -Xmx${mem}m -XX:ReservedCodeCacheSize=${codecache}m" echo "-Xms${mem}m -Xmx${mem}m -XX:ReservedCodeCacheSize=${codecache}m"
} }