ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-14572][DOC] Update config docs to allow -Xms in extraJavaOptions

Browse source 
## What changes were proposed in this pull request?
The configuration docs are updated to reflect the changes introduced with [SPARK-12384](https://issues.apache.org/jira/browse/SPARK-12384). This allows the user to specify initial heap memory settings through the extraJavaOptions for executor, driver and am.

## How was this patch tested?
The changes are tested in [SPARK-12384](https://issues.apache.org/jira/browse/SPARK-12384). This is just documenting the changes made.

Author: Dhruve Ashar <dhruveashar@gmail.com>

Closes #12333 from dhruve/doc/SPARK-14572.
This commit is contained in:
Dhruve Ashar 2016-04-14 10:29:14 -05:00 committed by Tom Graves
parent 3cf3db17b3
commit f83ba454a5
2 changed files with 10 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
docs/configuration.md
View file

@ -225,11 +225,14 @@ Apart from these, the following properties are also available, and may be useful
<td>(none)</td>
<td>
A string of extra JVM options to pass to the driver. For instance, GC settings or other logging.
Note that it is illegal to set maximum heap size (-Xmx) settings with this option. Maximum heap
size settings can be set with <code>spark.driver.memory</code> in the cluster mode and through
the <code>--driver-memory</code> command line option in the client mode.
<br /><em>Note:</em> In client mode, this config must not be set through the <code>SparkConf</code>
directly in your application, because the driver JVM has already started at that point.
Instead, please set this through the <code>--driver-java-options</code> command line option or in
your default properties file.</td>
your default properties file.
</td>
</tr>
<tr>
@ -269,9 +272,9 @@ Apart from these, the following properties are also available, and may be useful
<td>(none)</td>
<td>
A string of extra JVM options to pass to executors. For instance, GC settings or other logging.
Note that it is illegal to set Spark properties or heap size settings with this option. Spark
properties should be set using a SparkConf object or the spark-defaults.conf file used with the
spark-submit script. Heap size settings can be set with spark.executor.memory.
Note that it is illegal to set Spark properties or maximum heap size (-Xmx) settings with this
option. Spark properties should be set using a SparkConf object or the spark-defaults.conf file
used with the spark-submit script. Maximum heap size settings can be set with spark.executor.memory.
</td>
</tr>
<tr>

4
docs/running-on-yarn.md
View file

@ -342,7 +342,9 @@ If you need a reference to the proper location to put log files in the YARN so t
<td>(none)</td>
<td>
A string of extra JVM options to pass to the YARN Application Master in client mode.
In cluster mode, use <code>spark.driver.extraJavaOptions</code> instead.
In cluster mode, use <code>spark.driver.extraJavaOptions</code> instead. Note that it is illegal
to set maximum heap size (-Xmx) settings with this option. Maximum heap size settings can be set
with <code>spark.yarn.am.memory</code>
</td>
</tr>
<tr>