ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

Documenting the newly added spark properties.

Browse source
This commit is contained in:
Prashant Sharma 2013-11-26 20:44:47 +05:30
parent 560e44a8e1
commit dca946ff67
1 changed files with 22 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

23
docs/configuration.md
View file

@ -275,11 +275,32 @@ Apart from these, the following properties are also available, and may be useful
</tr> </tr>
<tr> <tr>
<td>spark.akka.timeout</td> <td>spark.akka.timeout</td>
<td>20</td> <td>60</td>
<td> <td>
Communication timeout between Spark nodes, in seconds. Communication timeout between Spark nodes, in seconds.
</td> </td>
</tr> </tr>
<tr>
<td>spark.akka.pauses</td>
<td>60</td>
<td>
Acceptable heart beat pause in seconds for akka, tune this if you expect GC pauses or network delays (reconnections) etc.
</td>
</tr>
<tr>
<td>spark.akka.failure-detector.threshold</td>
<td>12.0</td>
<td>
Tuning this is not recommended unless you are sure what you are doing. This maps to akka's `akka.remote.transport-failure-detector.threshold`. Usually having a larger value of `spark.akka.pauses` should suffice.
</td>
</tr>
<tr>
<td>spark.akka.heartbeat.interval</td>
<td>5</td>
<td>
A larger interval value in seconds reduces network overhead and a smaller value might be more informative for akka's failure detector. Tune this in combination of `spark.akka.pauses` and `spark.akka.failure-detector.threshold` if you need to.
</td>
</tr>
<tr> <tr>
<td>spark.driver.host</td> <td>spark.driver.host</td>
<td>(local hostname)</td> <td>(local hostname)</td>