Prashant Sharma
2bc348e92c
Linking custom receiver guide
2013-08-23 09:44:02 +05:30
Prashant Sharma
3049415e24
Corrections in documentation comment
2013-08-23 09:40:28 +05:30
Prashant Sharma
39a1d58da4
Improved documentation for spark custom receiver
2013-08-23 09:38:50 +05:30
Matei Zaharia
215c13dd41
Fix code style and a nondeterministic RDD issue in ALS
2013-08-22 16:13:46 -07:00
Matei Zaharia
46ea0c1b47
Merge pull request #814 from holdenk/master
...
Create less instances of the random class during ALS initialization.
2013-08-22 15:57:28 -07:00
Matei Zaharia
9ac3d62cac
Merge pull request #856 from jey/sbt-fix-hadoop-0.23.9
...
Re-add removed dependency to fix build under Hadoop 0.23.9
2013-08-22 15:51:10 -07:00
Jey Kottalam
281b6c5f28
Re-add removed dependency on 'commons-daemon'
...
Fixes SBT build under Hadoop 0.23.9 and 2.0.4
2013-08-22 15:45:45 -07:00
Matei Zaharia
ae8ba83ef2
Merge pull request #855 from jey/update-build-docs
...
Update build docs
2013-08-22 10:14:54 -07:00
Matei Zaharia
8a36fd09dd
Merge pull request #854 from markhamstra/pomUpdate
...
Synced sbt and maven builds to use the same dependencies, etc.
2013-08-22 10:13:35 -07:00
Matei Zaharia
c2d00f12e2
Merge pull request #832 from alig/coalesce
...
Coalesced RDD with locality
2013-08-22 10:13:03 -07:00
Jey Kottalam
9a90667d09
Increase ReservedCodeCacheSize to 256m
2013-08-21 21:15:28 -07:00
Jey Kottalam
0087b43e9c
Use Hadoop 1.2.1 in application example
2013-08-21 21:15:00 -07:00
Jey Kottalam
54e9379de2
Revert "Allow build configuration to be set in conf/spark-env.sh"
...
This reverts commit 66e7a38a32
.
2013-08-21 21:13:34 -07:00
Matei Zaharia
e6d66c8abd
Merge pull request #853 from AndreSchumacher/double_rdd
...
Implementing SPARK-838: Add DoubleRDDFunctions methods to PySpark
2013-08-21 17:44:31 -07:00
Jey Kottalam
f9cc1fbf27
Remove references to unsupported Hadoop versions
2013-08-21 17:14:36 -07:00
Andre Schumacher
76077bf9f4
Implementing SPARK-838: Add DoubleRDDFunctions methods to PySpark
2013-08-21 17:05:58 -07:00
Patrick Wendell
c02585ea13
Make initial connection failure message less daunting.
...
Right now it seems like something has gone wrong when this message is printed out.
Instead, this is a normal condition. So I changed the message a bit.
2013-08-21 15:45:45 -07:00
Patrick Wendell
6be6b71c8c
Merge branch 'master' into ec2-updates
...
Conflicts:
ec2/spark_ec2.py
2013-08-21 15:34:31 -07:00
Jey Kottalam
4d737b6d32
Example should make sense
2013-08-21 15:03:37 -07:00
Jey Kottalam
6585f49841
Update build docs
2013-08-21 14:51:56 -07:00
Jey Kottalam
66e7a38a32
Allow build configuration to be set in conf/spark-env.sh
2013-08-21 14:25:53 -07:00
Mark Hamstra
ff6f1b0500
Synced sbt and maven builds
2013-08-21 13:50:24 -07:00
Jey Kottalam
31644a011c
Use "hadoop.version" property when specifying Hadoop YARN version too
2013-08-21 13:24:28 -07:00
Jey Kottalam
9c6f8df30f
Update jekyll plugin to match docs/README.md
2013-08-21 12:57:56 -07:00
Matei Zaharia
111b2741fd
Change default SPARK_HADOOP_VERSION in make-distribution.sh too
2013-08-21 11:54:10 -07:00
Reynold Xin
8e3ea4c7db
Merge branch 'master' of github.com:mesos/spark
2013-08-21 11:38:51 -07:00
Reynold Xin
af602ba9d3
Downgraded default build hadoop version to 1.0.4.
2013-08-21 11:38:24 -07:00
Matei Zaharia
53b1c30607
Update docs for Spark UI port
2013-08-20 22:57:11 -07:00
Patrick Wendell
51a1a0c602
Bump spark version
2013-08-20 22:14:52 -07:00
Reynold Xin
2905611c13
Merge pull request #851 from markhamstra/MutablePairTE
...
Removed meaningless types
2013-08-20 17:36:14 -07:00
Mark Hamstra
5eea613ec0
Removed meaningless types
2013-08-20 16:49:18 -07:00
Ali Ghodsi
f20ed14e87
Merged in from upstream to use TaskLocation instead of strings
2013-08-20 16:21:43 -07:00
Ali Ghodsi
5cd21c4195
added curly braces to make the code more consistent
2013-08-20 16:16:05 -07:00
Ali Ghodsi
db4bc55bef
indent
2013-08-20 16:16:05 -07:00
Ali Ghodsi
c0942a710f
Bug in test fixed
2013-08-20 16:16:05 -07:00
Ali Ghodsi
5db41919b5
Added a test to make sure no locality preferences are ignored
2013-08-20 16:16:05 -07:00
Ali Ghodsi
7b123b3126
Simpler code
2013-08-20 16:16:05 -07:00
Ali Ghodsi
9192c358e4
simpler code
2013-08-20 16:16:05 -07:00
Ali Ghodsi
a75a64eade
Fixed almost all of Matei's feedback
2013-08-20 16:16:05 -07:00
Ali Ghodsi
f1c853d76d
fixed Matei's comments
2013-08-20 16:16:04 -07:00
Ali Ghodsi
890ea6ba79
making CoalescedRDDPartition public
2013-08-20 16:16:04 -07:00
Ali Ghodsi
d6b6c680be
comment in the test to make it more understandable
2013-08-20 16:16:04 -07:00
Ali Ghodsi
b69e7166ba
Coalescer now uses current preferred locations for derived RDDs. Made run() in DAGScheduler thread safe and added a method to be able to ask it for preferred locations. Added a similar method that wraps the former inside SparkContext.
2013-08-20 16:16:04 -07:00
Ali Ghodsi
3b5bb8a4ae
added one test that will test a future functionality
2013-08-20 16:13:37 -07:00
Ali Ghodsi
33a0f59354
Added error messages to the tests to make failed tests less cryptic
2013-08-20 16:13:37 -07:00
Ali Ghodsi
abcefb3858
fixed matei's comments
2013-08-20 16:13:37 -07:00
Ali Ghodsi
35537e6341
Made a function object that returns the coalesced groups
2013-08-20 16:13:37 -07:00
Ali Ghodsi
339598c080
several of Reynold's suggestions implemented
2013-08-20 16:13:37 -07:00
Ali Ghodsi
02d6464f2f
space removed
2013-08-20 16:13:37 -07:00
Ali Ghodsi
4f99be1ffd
use count rather than foreach
2013-08-20 16:13:37 -07:00