ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-35424][SHUFFLE] Remove some useless code in the ExternalBlockHandler

Browse source 
### What changes were proposed in this pull request?

Remove some useless code in the ExternalBlockHandler.

### Why are the changes needed?
There is some useless code in the ExternalBlockHandler, so we may remove it.

### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?

Exist unittests.

Closes #32571 from weixiuli/SPARK-35424.

Authored-by: weixiuli <weixiuli@jd.com>
Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
This commit is contained in:
weixiuli 2021-05-20 19:03:14 +09:00 committed by Hyukjin Kwon
parent e170e63955
commit 4869e437da
1 changed files with 0 additions and 15 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/ExternalBlockHandler.java
View file

@ -42,7 +42,6 @@ import org.apache.spark.network.client.TransportClient;
import org.apache.spark.network.server.OneForOneStreamManager;
import org.apache.spark.network.server.RpcHandler;
import org.apache.spark.network.server.StreamManager;
import org.apache.spark.network.shuffle.ExternalShuffleBlockResolver.AppExecId;
import org.apache.spark.network.shuffle.protocol.*;
import static org.apache.spark.network.util.NettyUtils.getRemoteAddress;
import org.apache.spark.network.util.TransportConf;
@ -242,20 +241,6 @@ public class ExternalBlockHandler extends RpcHandler {
blockManager.executorRemoved(executorId, appId);
}
/**
* Register an (application, executor) with the given shuffle info.
*
* The "re-" is meant to highlight the intended use of this method -- when this service is
* restarted, this is used to restore the state of executors from before the restart. Normal
* registration will happen via a message handled in receive()
*
* @param appExecId
* @param executorInfo
*/
public void reregisterExecutor(AppExecId appExecId, ExecutorShuffleInfo executorInfo) {
blockManager.registerExecutor(appExecId.appId, appExecId.execId, executorInfo);
}
public void close() {
blockManager.close();
}