ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-28634][YARN] Ignore kerberos login config in client mode AM

Browse source 
This change makes the client mode AM ignore any login configuration,
which is now always handled by the driver. The previous code tried
to achieve that by modifying the configuration visible to the AM, but
that missed the case where old configuration names were being used.

Tested in real cluster with reproduction provided in the bug.

Closes #25467 from vanzin/SPARK-28634.

Authored-by: Marcelo Vanzin <vanzin@cloudera.com>
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
This commit is contained in:
Marcelo Vanzin 2019-08-19 11:06:02 -07:00
parent 1de4a22c52
commit 5f6eb5d20d
2 changed files with 5 additions and 10 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
View file

@ -851,7 +851,9 @@ object ApplicationMaster extends Logging {
master = new ApplicationMaster(amArgs, sparkConf, yarnConf)
val ugi = sparkConf.get(PRINCIPAL) match {
case Some(principal) =>
// We only need to log in with the keytab in cluster mode. In client mode, the driver
// handles the user keytab.
case Some(principal) if amArgs.userClass != null =>
val originalCreds = UserGroupInformation.getCurrentUser().getCredentials()
SparkHadoopUtil.get.loginUserFromKeytab(principal, sparkConf.get(KEYTAB).orNull)
val newUGI = UserGroupInformation.getCurrentUser()

11
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
View file

@ -767,15 +767,8 @@ private[spark] class Client(
val props = confToProperties(sparkConf)
// If propagating the keytab to the AM, override the keytab name with the name of the
// distributed file. Otherwise remove princpal/keytab from the conf, so they're not seen
// by the AM at all.
amKeytabFileName match {
case Some(kt) =>
props.setProperty(KEYTAB.key, kt)
case None =>
props.remove(PRINCIPAL.key)
props.remove(KEYTAB.key)
}
// distributed file.
amKeytabFileName.foreach { kt => props.setProperty(KEYTAB.key, kt) }
writePropertiesToArchive(props, SPARK_CONF_FILE, confStream)