Merge pull request #349 from CodingCat/support-worker_dir
add the comments about SPARK_WORKER_DIR this env variable seems to be forgotten in many cases we need to set this variable, e.g. in EC2, we have to move the large application log files from the EBS to the ephemeral storage
This commit is contained in:
commit
71fc113574
|
@ -18,4 +18,4 @@
|
|||
# - SPARK_WORKER_MEMORY, to set how much memory to use (e.g. 1000m, 2g)
|
||||
# - SPARK_WORKER_PORT / SPARK_WORKER_WEBUI_PORT
|
||||
# - SPARK_WORKER_INSTANCES, to set the number of worker processes per node
|
||||
|
||||
# - SPARK_WORKER_DIR, to set the working directory of worker processes
|
||||
|
|
Loading…
Reference in a new issue