[SPARK-11751] Doc describe error in the "Spark Streaming Programming Guide" page
In the **[Task Launching Overheads](http://spark.apache.org/docs/latest/streaming-programming-guide.html#task-launching-overheads)** section, >Task Serialization: Using Kryo serialization for serializing tasks can reduce the task sizes, and therefore reduce the time taken to send them to the slaves. as we known **Task Serialization** is configuration by **spark.closure.serializer** parameter, but currently only the Java serializer is supported. If we set **spark.closure.serializer** to **org.apache.spark.serializer.KryoSerializer**, then this will throw a exception. Author: yangping.wu <wyphao.2007@163.com> Closes #9734 from 397090770/397090770-patch-1.
This commit is contained in:
parent
fa13301ae4
commit
7276fa9aa9
|
@ -2001,8 +2001,7 @@ If the number of tasks launched per second is high (say, 50 or more per second),
|
|||
of sending out tasks to the slaves may be significant and will make it hard to achieve sub-second
|
||||
latencies. The overhead can be reduced by the following changes:
|
||||
|
||||
* **Task Serialization**: Using Kryo serialization for serializing tasks can reduce the task
|
||||
sizes, and therefore reduce the time taken to send them to the slaves.
|
||||
* **Task Serialization**: Using Kryo serialization for serializing tasks can reduce the task sizes, and therefore reduce the time taken to send them to the slaves. This is controlled by the ```spark.closure.serializer``` property. However, at this time, Kryo serialization cannot be enabled for closure serialization. This may be resolved in a future release.
|
||||
|
||||
* **Execution mode**: Running Spark in Standalone mode or coarse-grained Mesos mode leads to
|
||||
better task launch times than the fine-grained Mesos mode. Please refer to the
|
||||
|
|
Loading…
Reference in a new issue