3159ee085b
## What changes were proposed in this pull request? Hadoop 3 introduces HDFS federation. This means that multiple namespaces are allowed on the same HDFS cluster. In Spark, we need to ask the delegation token for all the namenodes (for each namespace), otherwise accessing any other namespace different from the default one (for which we already fetch the delegation token) fails. The PR adds the automatic discovery of all the namenodes related to all the namespaces available according to the configs in hdfs-site.xml. ## How was this patch tested? manual tests in dockerized env Author: Marco Gaido <marcogaido91@gmail.com> Closes #21216 from mgaido91/SPARK-24149. |
||
|---|---|---|
| .. | ||
| kubernetes | ||
| mesos | ||
| yarn | ||