ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-13652][CORE] Copy ByteBuffer in sendRpcSync as it will be recycled

Browse source 
## What changes were proposed in this pull request?

`sendRpcSync` should copy the response content because the underlying buffer will be recycled and reused.

## How was this patch tested?

Jenkins unit tests.

Author: Shixiong Zhu <shixiong@databricks.com>

Closes #11499 from zsxwing/SPARK-13652.
This commit is contained in:
Shixiong Zhu 2016-03-03 22:53:07 -08:00
parent f6ac7c30d4
commit 465c665db1
2 changed files with 11 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
common/network-common/src/main/java/org/apache/spark/network/client/RpcResponseCallback.java
View file

@ -24,7 +24,12 @@ import java.nio.ByteBuffer;
* failure.
*/
public interface RpcResponseCallback {
/** Successful serialized result from server. */
/**
* Successful serialized result from server.
*
* After `onSuccess` returns, `response` will be recycled and its content will become invalid.
* Please copy the content of `response` if you want to use it after `onSuccess` returns.
*/
void onSuccess(ByteBuffer response);
/** Exception either propagated from server or raised on client side. */

6
common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java
View file

@ -257,7 +257,11 @@ public class TransportClient implements Closeable {
sendRpc(message, new RpcResponseCallback() {
@Override
public void onSuccess(ByteBuffer response) {
result.set(response);
ByteBuffer copy = ByteBuffer.allocate(response.remaining());
copy.put(response);
// flip "copy" to make it readable
copy.flip();
result.set(copy);
}
@Override