c0507e0f75
### What changes were proposed in this pull request? When set `spark.yarn.jars=/xxx/xxx` which is just a no schema path, spark will throw a NullPointerException. The reason is hdfs will return null if pathFs.globStatus(path) is not exist, and spark just use `pathFs.globStatus(path).filter(_.isFile())` without check it. ### Why are the changes needed? Avoid NullPointerException. ### Does this PR introduce any user-facing change? Yes. User will get a FileNotFoundException instead NullPointerException when `spark.yarn.jars` does not have schema and not exists. ### How was this patch tested? Add UT. Closes #26462 from ulysses-you/check-yarn-jars-path-exist. Authored-by: ulysses <youxiduo@weidian.com> Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com> |
||
|---|---|---|
| .. | ||
| kubernetes | ||
| mesos | ||
| yarn | ||