ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-2945][YARN][Doc]add doc for spark.executor.instances

Browse source 
https://issues.apache.org/jira/browse/SPARK-2945

spark.executor.instances works. As this JIRA recommended, we should add docs for this common config.

Author: WangTaoTheTonic <wangtao111@huawei.com>

Closes #4350 from WangTaoTheTonic/SPARK-2945 and squashes the following commits:

4c3913a [WangTaoTheTonic] not compatible with dynamic allocation
5fa9c46 [WangTaoTheTonic] add doc for spark.executor.instances
This commit is contained in:
WangTaoTheTonic 2015-02-06 11:57:02 -08:00 committed by Andrew Or
parent af2a2a263a
commit d34f79c8db
1 changed files with 7 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
docs/running-on-yarn.md
View file

@ -104,6 +104,13 @@ Most of the configs are the same for Spark on YARN as for other deployment modes
Comma-separated list of files to be placed in the working directory of each executor. Comma-separated list of files to be placed in the working directory of each executor.
</td> </td>
</tr> </tr>
<tr>
<td><code>spark.executor.instances</code></td>
<td>2</td>
<td>
The number of executors. Note that this property is incompatible with <code>spark.dynamicAllocation.enabled</code>.
</td>
</tr>
<tr> <tr>
<td><code>spark.yarn.executor.memoryOverhead</code></td> <td><code>spark.yarn.executor.memoryOverhead</code></td>
<td>executorMemory * 0.07, with minimum of 384 </td> <td>executorMemory * 0.07, with minimum of 384 </td>