ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[MINOR][DOC] Minor doc change for YARN credential providers

Browse source 
## What changes were proposed in this pull request?

The configuration `spark.yarn.security.tokens.{service}.enabled` is deprecated. Now we should use `spark.yarn.security.credentials.{service}.enabled`. Some places in the doc is not updated yet.

## How was this patch tested?

N/A. Just doc change.

Please review http://spark.apache.org/contributing.html before opening a pull request.

Author: Liang-Chi Hsieh <viirya@gmail.com>

Closes #16444 from viirya/minor-credential-provider-doc.
This commit is contained in:
Liang-Chi Hsieh 2017-01-02 14:41:57 +00:00 committed by Sean Owen
parent 808b84e2de
commit 0ac2f1e71f
No known key found for this signature in database
GPG key ID: BEB3956D6717BDDC
2 changed files with 5 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
docs/running-on-yarn.md
View file

@ -506,7 +506,7 @@ Spark supports integrating with other security-aware services through Java Servi
`java.util.ServiceLoader`). To do that, implementations of `org.apache.spark.deploy.yarn.security.ServiceCredentialProvider`
should be available to Spark by listing their names in the corresponding file in the jar's
`META-INF/services` directory. These plug-ins can be disabled by setting
`spark.yarn.security.tokens.{service}.enabled` to `false`, where `{service}` is the name of
`spark.yarn.security.credentials.{service}.enabled` to `false`, where `{service}` is the name of
credential provider.
## Configuring the External Shuffle Service
@ -570,8 +570,8 @@ the Spark configuration must be set to disable token collection for the services
The Spark configuration must include the lines:
```
spark.yarn.security.tokens.hive.enabled false
spark.yarn.security.tokens.hbase.enabled false
spark.yarn.security.credentials.hive.enabled false
spark.yarn.security.credentials.hbase.enabled false
```
The configuration option `spark.yarn.access.namenodes` must be unset.

2
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/security/ConfigurableCredentialManager.scala
View file

@ -38,6 +38,8 @@ import org.apache.spark.util.Utils
*
* Also each credential provider is controlled by
* spark.yarn.security.credentials.{service}.enabled, it will not be loaded in if set to false.
* For example, Hive's credential provider [[HiveCredentialProvider]] can be enabled/disabled by
* the configuration spark.yarn.security.credentials.hive.enabled.
*/
private[yarn] final class ConfigurableCredentialManager(
sparkConf: SparkConf, hadoopConf: Configuration) extends Logging {