[SPARK-15761][MLLIB][PYSPARK] Load ipython when default python is Python3
## What changes were proposed in this pull request? I would like to use IPython with Python 3.5. It is annoying when it fails with IPython requires Python 2.7+; please install python2.7 or set PYSPARK_PYTHON when I have a version greater than 2.7 ## How was this patch tested It now works with IPython and Python3 Author: MechCoder <mks542@nyu.edu> Closes #13503 from MechCoder/spark-15761.
This commit is contained in:
parent
2075bf8ef6
commit
66283ee0b2
|
@ -50,9 +50,11 @@ if [[ -z "$PYSPARK_DRIVER_PYTHON" ]]; then
|
||||||
PYSPARK_DRIVER_PYTHON="${PYSPARK_PYTHON:-"$DEFAULT_PYTHON"}"
|
PYSPARK_DRIVER_PYTHON="${PYSPARK_PYTHON:-"$DEFAULT_PYTHON"}"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
WORKS_WITH_IPYTHON=$($DEFAULT_PYTHON -c 'import sys; print(sys.version_info >= (2, 7, 0))')
|
||||||
|
|
||||||
# Determine the Python executable to use for the executors:
|
# Determine the Python executable to use for the executors:
|
||||||
if [[ -z "$PYSPARK_PYTHON" ]]; then
|
if [[ -z "$PYSPARK_PYTHON" ]]; then
|
||||||
if [[ $PYSPARK_DRIVER_PYTHON == *ipython* && $DEFAULT_PYTHON != "python2.7" ]]; then
|
if [[ $PYSPARK_DRIVER_PYTHON == *ipython* && ! WORKS_WITH_IPYTHON ]]; then
|
||||||
echo "IPython requires Python 2.7+; please install python2.7 or set PYSPARK_PYTHON" 1>&2
|
echo "IPython requires Python 2.7+; please install python2.7 or set PYSPARK_PYTHON" 1>&2
|
||||||
exit 1
|
exit 1
|
||||||
else
|
else
|
||||||
|
|
Loading…
Reference in a new issue