spark-instrumented-optimizer/resource-managers
Marcelo Vanzin 78ecb6d457 [SPARK-24446][YARN] Properly quote library path for YARN.
Because the way YARN executes commands via bash -c, everything needs
to be quoted so that the whole command is fully contained inside a
bash string and is interpreted correctly when the string is read by
bash. This is a bit different than the quoting done when executing
things as if typing in a bash shell.

Tweaked unit tests to exercise the bad behavior, which would cause
existing tests to time out without the fix. Also tested on a real
cluster, verifying the shell script created by YARN to run the
container.

Author: Marcelo Vanzin <vanzin@cloudera.com>

Closes #21476 from vanzin/SPARK-24446.
2018-06-27 10:57:29 -07:00
..
kubernetes [SPARK-24534][K8S] Bypass non spark-on-k8s commands 2018-06-19 13:25:00 -07:00
mesos [SPARK-16630][YARN] Blacklist a node if executors won't launch on it 2018-06-21 09:17:18 -05:00
yarn [SPARK-24446][YARN] Properly quote library path for YARN. 2018-06-27 10:57:29 -07:00