70beb808e1
Otherwise the script will crash with
- Downloading boto...
Traceback (most recent call last):
File "ec2/spark_ec2.py", line 148, in <module>
setup_external_libs(external_libs)
File "ec2/spark_ec2.py", line 128, in setup_external_libs
if hashlib.md5(tar.read()).hexdigest() != lib["md5"]:
File "/usr/lib/python3.4/codecs.py", line 319, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte
In case of an utf8 env setting.
Author: Simon Hafner <hafnersimon@gmail.com>
Closes #7215 from reactormonk/branch-1.4 and squashes the following commits:
e86957a [Simon Hafner] [SPARK-8821] [EC2] Switched to binary mode
(cherry picked from commit
|
||
---|---|---|
.. | ||
deploy.generic/root/spark-ec2 | ||
README | ||
spark-ec2 | ||
spark_ec2.py |
This folder contains a script, spark-ec2, for launching Spark clusters on Amazon EC2. Usage instructions are available online at: http://spark.apache.org/docs/latest/ec2-scripts.html