spark-instrumented-optimizer/resource-managers
jerryshao 4239a1081a [SPARK-19021][YARN] Generailize HDFSCredentialProvider to support non HDFS security filesystems
Currently Spark can only get token renewal interval from security HDFS (hdfs://), if Spark runs with other security file systems like webHDFS (webhdfs://), wasb (wasb://), ADLS, it will ignore these tokens and not get token renewal intervals from these tokens. These will make Spark unable to work with these security clusters. So instead of only checking HDFS token, we should generalize to support different DelegationTokenIdentifier.

## How was this patch tested?

Manually verified in security cluster.

Author: jerryshao <sshao@hortonworks.com>

Closes #16432 from jerryshao/SPARK-19021.
2017-01-11 09:24:02 -06:00
..
mesos [SPARK-17931] Eliminate unnecessary task (de) serialization 2017-01-06 10:48:08 -06:00
yarn [SPARK-19021][YARN] Generailize HDFSCredentialProvider to support non HDFS security filesystems 2017-01-11 09:24:02 -06:00