ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/ec2
History
Nicholas Chammas f17d43b033 [SPARK-6219] [Build] Check that Python code compiles 
This PR expands the Python lint checks so that they check for obvious compilation errors in our Python code.

For example:

```
$ ./dev/lint-python
Python lint checks failed.
Compiling ./ec2/spark_ec2.py ...
  File "./ec2/spark_ec2.py", line 618
    return (master_nodes,, slave_nodes)
                         ^
SyntaxError: invalid syntax

./ec2/spark_ec2.py:618:25: E231 missing whitespace after ','
./ec2/spark_ec2.py:1117:101: E501 line too long (102 > 100 characters)
```

This PR also bumps up the version of `pep8`. It ignores new types of checks introduced by that version bump while fixing problems missed by the older version of `pep8` we were using.

Author: Nicholas Chammas <nicholas.chammas@gmail.com>

Closes #4941 from nchammas/compile-spark-ec2 and squashes the following commits:

75e31d8 [Nicholas Chammas] upgrade pep8 + check compile
b33651c [Nicholas Chammas] PEP8 line length
2015-03-19 12:46:10 -07:00
..
deploy.generic/root/spark-ec2 [SPARK-6402][DOC] - Remove some refererences to shark in docs and ec2 2015-03-19 08:02:06 -04:00
README Remove remaining references to incubation 2014-03-02 01:00:16 -08:00
spark-ec2 [SPARK-5434] [EC2] Preserve spaces in EC2 path 2015-01-28 12:56:03 -08:00
spark_ec2.py [SPARK-6219] [Build] Check that Python code compiles 2015-03-19 12:46:10 -07:00

README 

This folder contains a script, spark-ec2, for launching Spark clusters on
Amazon EC2. Usage instructions are available online at:

http://spark.apache.org/docs/latest/ec2-scripts.html