Aaron Davidson
3a04e76c89
Reynold's second round of comments
2013-09-05 21:43:26 -07:00
Ameet Talwalkar
d52edfa753
updated content
2013-09-05 21:06:50 -07:00
Matei Zaharia
699c331f2f
Merge pull request #891 from xiajunluan/SPARK-864
...
[SPARK-864]DAGScheduler Exception if we delete Worker and StandaloneExecutorBackend then add Worker
2013-09-05 20:21:53 -07:00
Aaron Davidson
4f2236a1c5
Add unit test and address comments
2013-09-05 18:06:30 -07:00
Aaron Davidson
1418d18af4
SPARK-821: Don't cache results when action run locally on driver
...
Caching the results of local actions (e.g., rdd.first()) causes the driver to
store entire partitions in its own memory, which may be highly constrained.
This patch simply makes the CacheManager avoid caching the result of all locally-run computations.
2013-09-05 15:34:42 -07:00
Andrew xia
7c15e3c5de
Fix bug SPARK-864
2013-09-05 15:56:11 +08:00
Patrick Wendell
5c7494d7c1
Merge pull request #893 from ilikerps/master
...
SPARK-884: Add unit test to validate Spark JSON output
2013-09-04 22:47:03 -07:00
Aaron Davidson
714e7f9e32
Fix line over 100 chars
2013-09-04 22:40:08 -07:00
Aaron Davidson
37db141aef
Address Patrick's comments
2013-09-04 21:34:20 -07:00
Matei Zaharia
a54786678f
Merge pull request #894 from c0s/master
...
Updating assembly README to reflect recent changes in the build.
2013-09-04 21:11:56 -07:00
Konstantin Boudnik
7c7c7e10ca
Updating assembly README to reflect recent changes in the build.
2013-09-04 20:54:35 -07:00
Aaron Davidson
9e6f2b6822
SPARK-884: Add unit test to validate Spark JSON output
...
This unit test simply validates that the outputs of
the JsonProtocol methods are syntactically valid JSON.
2013-09-04 15:26:46 -07:00
Mridul Muralidharan
1e2474b814
Address review comments - rename toHash to nonNegativeHash
2013-09-04 07:46:46 +05:30
Mridul Muralidharan
b3a82b7df3
Fix hash bug - caused failure after 35k stages, sigh
2013-09-04 07:02:25 +05:30
Mark Hamstra
c9bc8af3d1
Removed repetative import; fixes hidden definition compiler warning.
2013-09-03 15:25:20 -07:00
Patrick Wendell
c592a3c9b9
Minor spacing fix
2013-09-03 14:39:11 -07:00
Patrick Wendell
19f70273d2
Merge pull request #878 from tgravescs/yarnUILink
...
Link the Spark UI up to the Yarn UI
2013-09-03 14:29:10 -07:00
Matei Zaharia
68df2464d1
Merge pull request #889 from alig/master
...
Return the port the WebUI is bound to (useful if port 0 was used)
2013-09-03 13:01:17 -07:00
Y.CORP.YAHOO.COM\tgraves
41c1b5b9a0
Update based on review comments. Change function to prependBaseUri and fix formatting.
2013-09-03 14:46:51 -05:00
Y.CORP.YAHOO.COM\tgraves
c8cc276110
Review comment changes and update to org.apache packaging
2013-09-03 10:50:21 -05:00
Y.CORP.YAHOO.COM\tgraves
547fc4a412
Merge remote-tracking branch 'mesos/master' into yarnUILink
...
Conflicts:
core/src/main/scala/org/apache/spark/ui/UIUtils.scala
core/src/main/scala/org/apache/spark/ui/jobs/PoolTable.scala
core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala
docs/running-on-yarn.md
2013-09-03 08:36:59 -05:00
Ali Ghodsi
b25918d841
Merge branch 'master' of https://github.com/alig/spark
...
Conflicts:
core/src/main/scala/org/apache/spark/deploy/master/Master.scala
2013-09-03 00:56:12 -07:00
Ali Ghodsi
bd0788505f
Using configured akka timeouts
2013-09-03 00:50:35 -07:00
Ali Ghodsi
cbfef9b3ff
Sort order of imports to match project guidelines
2013-09-02 19:33:55 -07:00
Ali Ghodsi
36d8fca2cc
Reynold's comment fixed
2013-09-02 19:31:09 -07:00
Ali Ghodsi
e452bd6d77
Brushing the code up slightly
2013-09-02 19:04:08 -07:00
Matei Zaharia
a106ed8b97
Merge remote-tracking branch 'old/master'
2013-09-02 18:38:12 -07:00
Matei Zaharia
59218bdd49
Add Apache parent POM
2013-09-02 18:34:03 -07:00
Ali Ghodsi
cf7b115496
Enabling getting the actual WEBUI port
2013-09-02 18:21:21 -07:00
Matei Zaharia
d3dd48f8b3
Merge pull request #887 from mateiz/misc-fixes
...
Miscellaneous fixes for 0.8
2013-09-02 16:44:54 -07:00
Matei Zaharia
f586c8ef38
Updated LICENSE with third-party licenses
2013-09-02 16:43:06 -07:00
Matei Zaharia
4cc27d962b
Add Apache incubator notice to README
2013-09-02 14:34:09 -07:00
Matei Zaharia
2615cad30b
Some doc improvements
...
- List higher-level projects that run on Spark
- Tweak CSS
2013-09-02 13:35:28 -07:00
Matei Zaharia
12b2f1f9c9
Add missing license headers found with RAT
2013-09-02 12:23:03 -07:00
Matei Zaharia
2ba695292a
Exclude some private modules in epydoc
2013-09-02 12:22:52 -07:00
Matei Zaharia
636fc0c89e
Merge pull request #886 from mateiz/codec
...
Fix spark.io.compression.codec and change default codec to LZF
2013-09-02 11:20:39 -07:00
Matei Zaharia
246bf67f58
Fix test
2013-09-02 10:57:34 -07:00
Matei Zaharia
9329a7d4cd
Fix spark.io.compression.codec and change default codec to LZF
2013-09-02 10:15:22 -07:00
Matei Zaharia
d9a53b94f8
Merge pull request #885 from mateiz/win-py
...
Allow PySpark to run on Windows
2013-09-01 22:12:30 -07:00
Matei Zaharia
9ee1e9db2e
Doc improvements
2013-09-01 22:12:03 -07:00
Matei Zaharia
141f54279e
Further fixes to get PySpark to work on Windows
2013-09-02 01:19:29 +00:00
Matei Zaharia
6550e5e60c
Allow PySpark to launch worker.py directly on Windows
2013-09-01 18:06:15 -07:00
Matei Zaharia
3c520fea77
Merge pull request #884 from mateiz/win-fixes
...
Run script fixes for Windows after package & assembly change
2013-09-01 17:26:55 -07:00
Matei Zaharia
3db404a43a
Run script fixes for Windows after package & assembly change
2013-09-01 23:45:57 +00:00
Matei Zaharia
2ce200bf7f
Merge remote-tracking branch 'old/master'
2013-09-01 14:57:27 -07:00
Matei Zaharia
f957c26fa2
Merge pull request #882 from mateiz/package-rename
...
Rename spark package to org.apache.spark
2013-09-01 14:53:57 -07:00
Matei Zaharia
0a8cc30921
Move some classes to more appropriate packages:
...
* RDD, *RDDFunctions -> org.apache.spark.rdd
* Utils, ClosureCleaner, SizeEstimator -> org.apache.spark.util
* JavaSerializer, KryoSerializer -> org.apache.spark.serializer
2013-09-01 14:13:16 -07:00
Matei Zaharia
5b4dea2143
More fixes
2013-09-01 14:13:16 -07:00
Matei Zaharia
5701eb92c7
Fix some URLs
2013-09-01 14:13:16 -07:00
Matei Zaharia
debcf24389
Fix over-zealous find-and-replace in HTML
2013-09-01 14:13:16 -07:00