ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-2435] Add shutdown hook to pyspark

Browse source 
Author: Matthew Farrellee <matt@redhat.com>

Closes #2183 from mattf/SPARK-2435 and squashes the following commits:

ee0ee99 [Matthew Farrellee] [SPARK-2435] Add shutdown hook to pyspark
This commit is contained in:
Matthew Farrellee 2014-09-03 19:37:37 -07:00 committed by Josh Rosen
parent c5cbc49233
commit 7c6e71f05f
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
python/pyspark/shell.py
View file

@ -28,6 +28,7 @@ if sys.version_info[0] != 2:
sys.exit(1)
import atexit
import os
import platform
import pyspark
@ -42,6 +43,7 @@ if os.environ.get("SPARK_EXECUTOR_URI"):
SparkContext.setSystemProperty("spark.executor.uri", os.environ["SPARK_EXECUTOR_URI"])
sc = SparkContext(appName="PySparkShell", pyFiles=add_files)
atexit.register(lambda: sc.stop())
print("""Welcome to
____ __