spark-instrumented-optimizer/core
Alessandro Bellina 79a650494f [SPARK-26895][CORE] prepareSubmitEnvironment should be called within doAs for proxy users
## What changes were proposed in this pull request?

`prepareSubmitEnvironment` performs globbing that will fail in the case where a proxy user (`--proxy-user`) doesn't have permission to the file. This is a bug also with 2.3, so we should backport, as currently you can't launch an application that for instance is passing a file under `--archives`, and that file is owned by the target user.

The solution is to call `prepareSubmitEnvironment` within a doAs context if proxying.

## How was this patch tested?

Manual tests running with `--proxy-user` and `--archives`, before and after, showing that the globbing is successful when the resource is owned by the target user.

I've looked at writing unit tests, but I am not sure I can do that cleanly (perhaps with a custom FileSystem). Open to ideas.

Please review http://spark.apache.org/contributing.html before opening a pull request.

Closes #23806 from abellina/SPARK-26895_prepareSubmitEnvironment_from_doAs.

Lead-authored-by: Alessandro Bellina <abellina@gmail.com>
Co-authored-by: Alessandro Bellina <abellina@yahoo-inc.com>
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
2019-02-22 11:15:20 -08:00
..
benchmarks [SPARK-26816][CORE][TEST] Add XORShiftRandom Benchmark 2019-02-10 13:52:24 -08:00
src [SPARK-26895][CORE] prepareSubmitEnvironment should be called within doAs for proxy users 2019-02-22 11:15:20 -08:00
pom.xml [SPARK-26254][CORE] Extract Hive + Kafka dependencies from Core. 2019-01-25 10:36:00 -08:00