ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[MINOR][DOC] Minor doc change for YARN credential providers

Browse source 
## What changes were proposed in this pull request?

The configuration `spark.yarn.security.tokens.{service}.enabled` is deprecated. Now we should use `spark.yarn.security.credentials.{service}.enabled`. Some places in the doc is not updated yet.

## How was this patch tested?

N/A. Just doc change.

Please review http://spark.apache.org/contributing.html before opening a pull request.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #16444 from viirya/minor-credential-provider-doc.
This commit is contained in:
Liang-Chi Hsieh 2017-01-02 14:41:57 +00:00 committed by Sean Owen
parent 808b84e2de
commit 0ac2f1e71f
No known key found for this signature in database
GPG key ID: BEB3956D6717BDDC
2 changed files with 5 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/running-on-yarn.md
View file

@ -506,7 +506,7 @@ Spark supports integrating with other security-aware services through Java Servi
`java.util.ServiceLoader`). To do that, implementations of `org.apache.spark.deploy.yarn.security.ServiceCredentialProvider` `java.util.ServiceLoader`). To do that, implementations of `org.apache.spark.deploy.yarn.security.ServiceCredentialProvider`
should be available to Spark by listing their names in the corresponding file in the jar's should be available to Spark by listing their names in the corresponding file in the jar's
`META-INF/services` directory. These plug-ins can be disabled by setting `META-INF/services` directory. These plug-ins can be disabled by setting
`spark.yarn.security.tokens.{service}.enabled` to `false`, where `{service}` is the name of `spark.yarn.security.credentials.{service}.enabled` to `false`, where `{service}` is the name of
credential provider. credential provider.
## Configuring the External Shuffle Service ## Configuring the External Shuffle Service
@ -570,8 +570,8 @@ the Spark configuration must be set to disable token collection for the services
The Spark configuration must include the lines: The Spark configuration must include the lines:
``` ```
spark.yarn.security.tokens.hive.enabled false spark.yarn.security.credentials.hive.enabled false
spark.yarn.security.tokens.hbase.enabled false spark.yarn.security.credentials.hbase.enabled false
``` ```
The configuration option `spark.yarn.access.namenodes` must be unset. The configuration option `spark.yarn.access.namenodes` must be unset.

2
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/ConfigurableCredentialManager.scala
View file

@ -38,6 +38,8 @@ import org.apache.spark.util.Utils
* *
* Also each credential provider is controlled by * Also each credential provider is controlled by
* spark.yarn.security.credentials.{service}.enabled, it will not be loaded in if set to false. * spark.yarn.security.credentials.{service}.enabled, it will not be loaded in if set to false.
* For example, Hive's credential provider [[HiveCredentialProvider]] can be enabled/disabled by
* the configuration spark.yarn.security.credentials.hive.enabled.
*/ */
private[yarn] final class ConfigurableCredentialManager( private[yarn] final class ConfigurableCredentialManager(
sparkConf: SparkConf, hadoopConf: Configuration) extends Logging { sparkConf: SparkConf, hadoopConf: Configuration) extends Logging {