[SPARK-13414][MESOS] Allow multiple dispatchers to be launched.
## What changes were proposed in this pull request? Users might want to start multiple mesos dispatchers, as each dispatcher can potentially be part of different roles and used for multi-tenancy. To allow multiple Mesos dispatchers to be launched, we need to be able to specify a instance number when starting the dispatcher daemon. ## How was the this patch tested? Manual testing Author: Timothy Chen <tnachen@gmail.com> Closes #11281 from tnachen/multiple_cluster_dispatchers.
This commit is contained in:
parent
6ce7c481dc
commit
a4a081d1df
|
@ -37,5 +37,8 @@ if [ "$SPARK_MESOS_DISPATCHER_HOST" = "" ]; then
|
|||
SPARK_MESOS_DISPATCHER_HOST=`hostname`
|
||||
fi
|
||||
|
||||
if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
|
||||
SPARK_MESOS_DISPATCHER_NUM=1
|
||||
fi
|
||||
|
||||
"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher 1 --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
|
||||
"${SPARK_HOME}/sbin"/spark-daemon.sh start org.apache.spark.deploy.mesos.MesosClusterDispatcher $SPARK_MESOS_DISPATCHER_NUM --host $SPARK_MESOS_DISPATCHER_HOST --port $SPARK_MESOS_DISPATCHER_PORT "$@"
|
||||
|
|
|
@ -24,5 +24,10 @@ fi
|
|||
|
||||
. "${SPARK_HOME}/sbin/spark-config.sh"
|
||||
|
||||
"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher 1
|
||||
if [ "$SPARK_MESOS_DISPATCHER_NUM" = "" ]; then
|
||||
SPARK_MESOS_DISPATCHER_NUM=1
|
||||
fi
|
||||
|
||||
"${SPARK_HOME}/sbin"/spark-daemon.sh stop org.apache.spark.deploy.mesos.MesosClusterDispatcher \
|
||||
$SPARK_MESOS_DISPATCHER_NUM
|
||||
|
||||
|
|
Loading…
Reference in a new issue