ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-13414][MESOS] Allow multiple dispatchers to be launched.

Browse source 
## What changes were proposed in this pull request?

Users might want to start multiple mesos dispatchers, as each dispatcher can potentially be part of different roles and used for multi-tenancy.

To allow multiple Mesos dispatchers to be launched, we need to be able to specify a instance number when starting the dispatcher daemon.

## How was the this patch tested?

Manual testing

Author: Timothy Chen <tnachen@gmail.com>

Closes #11281 from tnachen/multiple_cluster_dispatchers.
This commit is contained in:
Timothy Chen 2016-02-20 12:58:47 -08:00 committed by Reynold Xin
parent 6ce7c481dc
commit a4a081d1df
2 changed files with 10 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
sbin/start-mesos-dispatcher.sh
View file

@ -37,5 +37,8 @@ if [ "$SPARK_MESOS_DISPATCHER_HOST" = "" ]; then
SPARK_MESOS_DISPATCHER_HOST=`hostname`
fi
if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
SPARK_MESOS_DISPATCHER_NUM=1
fi
"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher 1 --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher $SPARK_MESOS_DISPATCHER_NUM --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"

7
sbin/stop-mesos-dispatcher.sh
View file

@ -24,5 +24,10 @@ fi
. "${SPARK_HOME}/sbin/spark-config.sh"
"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher 1
if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
SPARK_MESOS_DISPATCHER_NUM=1
fi
"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher \
$SPARK_MESOS_DISPATCHER_NUM