ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-3399][PySpark] Test for PySpark should ignore HADOOP_CONF_DIR and YARN_CONF_DIR

Browse source 
Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp>

Closes #2270 from sarutak/SPARK-3399 and squashes the following commits:

7613be6 [Kousuke Saruta] Modified pyspark script to ignore environment variables YARN_CONF_DIR and HADOOP_CONF_DIR while testing
This commit is contained in:
Kousuke Saruta 2014-09-05 11:07:00 -07:00 committed by Josh Rosen
parent 62c5576099
commit 7ff8c45d71
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
bin/pyspark
View file

@ -85,6 +85,8 @@ export PYSPARK_SUBMIT_ARGS
# For pyspark tests
if [[ -n "$SPARK_TESTING" ]]; then
unset YARN_CONF_DIR
unset HADOOP_CONF_DIR
if [[ -n "$PYSPARK_DOC_TEST" ]]; then
exec "$PYSPARK_PYTHON" -m doctest $1
else