ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-4183] Enable NettyBlockTransferService by default

Browse source 
Note that we're turning this on for at least the first part of the QA period as a trial. We want to enable this (and deprecate the NioBlockTransferService) as soon as possible in the hopes that NettyBlockTransferService will be more stable and easier to maintain. We will turn it off if we run into major issues.

Author: Aaron Davidson <aaron@databricks.com>

Closes #3049 from aarondav/enable-netty and squashes the following commits:

bb981cc [Aaron Davidson] [SPARK-4183] Enable NettyBlockTransferService by default
This commit is contained in:
Aaron Davidson 2014-11-01 13:15:24 -07:00 committed by Patrick Wendell
parent ebd6480587
commit 1ae51f6dc7
1 changed files with 10 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
docs/configuration.md
View file

@ -371,6 +371,16 @@ Apart from these, the following properties are also available, and may be useful
map-side aggregation and there are at most this many reduce partitions.
</td>
</tr>
<tr>
<td><code>spark.shuffle.blockTransferService</code></td>
<td>netty</td>
<td>
Implementation to use for transferring shuffle and cached blocks between executors. There
are two implementations available: <code>netty</code> and <code>nio</code>. Netty-based
block transfer is intended to be simpler but equally efficient and is the default option
starting in 1.2.
</td>
</tr>
</table>
#### Spark UI