Matei Zaharia
d282c1ebbb
Merge pull request #860 from jey/sbt-ide-fixes
...
Fix IDE project generation under SBT
2013-08-23 11:20:20 -07:00
Jey Kottalam
a9db1b7b6e
Upgrade SBT IDE project generators
2013-08-23 10:27:18 -07:00
Jey Kottalam
b7f9e6374a
Fix SBT generation of IDE project files
2013-08-23 10:26:37 -07:00
Ian Buss
d7f18e3d27
Pass SBT_OPTS environment through to sbt_launcher
2013-08-23 09:50:23 +01:00
Matei Zaharia
5a6ac12840
Merge pull request #701 from ScrapCodes/documentation-suggestions
...
Documentation suggestions for spark streaming.
2013-08-22 22:08:03 -07:00
Prashant Sharma
2bc348e92c
Linking custom receiver guide
2013-08-23 09:44:02 +05:30
Prashant Sharma
3049415e24
Corrections in documentation comment
2013-08-23 09:40:28 +05:30
Prashant Sharma
39a1d58da4
Improved documentation for spark custom receiver
2013-08-23 09:38:50 +05:30
Matei Zaharia
215c13dd41
Fix code style and a nondeterministic RDD issue in ALS
2013-08-22 16:13:46 -07:00
Matei Zaharia
46ea0c1b47
Merge pull request #814 from holdenk/master
...
Create less instances of the random class during ALS initialization.
2013-08-22 15:57:28 -07:00
Matei Zaharia
9ac3d62cac
Merge pull request #856 from jey/sbt-fix-hadoop-0.23.9
...
Re-add removed dependency to fix build under Hadoop 0.23.9
2013-08-22 15:51:10 -07:00
Jey Kottalam
281b6c5f28
Re-add removed dependency on 'commons-daemon'
...
Fixes SBT build under Hadoop 0.23.9 and 2.0.4
2013-08-22 15:45:45 -07:00
Matei Zaharia
ae8ba83ef2
Merge pull request #855 from jey/update-build-docs
...
Update build docs
2013-08-22 10:14:54 -07:00
Matei Zaharia
8a36fd09dd
Merge pull request #854 from markhamstra/pomUpdate
...
Synced sbt and maven builds to use the same dependencies, etc.
2013-08-22 10:13:35 -07:00
Matei Zaharia
c2d00f12e2
Merge pull request #832 from alig/coalesce
...
Coalesced RDD with locality
2013-08-22 10:13:03 -07:00
Jey Kottalam
9a90667d09
Increase ReservedCodeCacheSize to 256m
2013-08-21 21:15:28 -07:00
Jey Kottalam
0087b43e9c
Use Hadoop 1.2.1 in application example
2013-08-21 21:15:00 -07:00
Jey Kottalam
54e9379de2
Revert "Allow build configuration to be set in conf/spark-env.sh"
...
This reverts commit 66e7a38a32
.
2013-08-21 21:13:34 -07:00
Matei Zaharia
e6d66c8abd
Merge pull request #853 from AndreSchumacher/double_rdd
...
Implementing SPARK-838: Add DoubleRDDFunctions methods to PySpark
2013-08-21 17:44:31 -07:00
Jey Kottalam
f9cc1fbf27
Remove references to unsupported Hadoop versions
2013-08-21 17:14:36 -07:00
Andre Schumacher
76077bf9f4
Implementing SPARK-838: Add DoubleRDDFunctions methods to PySpark
2013-08-21 17:05:58 -07:00
Patrick Wendell
c02585ea13
Make initial connection failure message less daunting.
...
Right now it seems like something has gone wrong when this message is printed out.
Instead, this is a normal condition. So I changed the message a bit.
2013-08-21 15:45:45 -07:00
Patrick Wendell
6be6b71c8c
Merge branch 'master' into ec2-updates
...
Conflicts:
ec2/spark_ec2.py
2013-08-21 15:34:31 -07:00
Jey Kottalam
4d737b6d32
Example should make sense
2013-08-21 15:03:37 -07:00
Jey Kottalam
6585f49841
Update build docs
2013-08-21 14:51:56 -07:00
Jey Kottalam
66e7a38a32
Allow build configuration to be set in conf/spark-env.sh
2013-08-21 14:25:53 -07:00
Mark Hamstra
ff6f1b0500
Synced sbt and maven builds
2013-08-21 13:50:24 -07:00
Jey Kottalam
31644a011c
Use "hadoop.version" property when specifying Hadoop YARN version too
2013-08-21 13:24:28 -07:00
Jey Kottalam
9c6f8df30f
Update jekyll plugin to match docs/README.md
2013-08-21 12:57:56 -07:00
Matei Zaharia
111b2741fd
Change default SPARK_HADOOP_VERSION in make-distribution.sh too
2013-08-21 11:54:10 -07:00
Reynold Xin
8e3ea4c7db
Merge branch 'master' of github.com:mesos/spark
2013-08-21 11:38:51 -07:00
Reynold Xin
af602ba9d3
Downgraded default build hadoop version to 1.0.4.
2013-08-21 11:38:24 -07:00
Matei Zaharia
53b1c30607
Update docs for Spark UI port
2013-08-20 22:57:11 -07:00
Patrick Wendell
51a1a0c602
Bump spark version
2013-08-20 22:14:52 -07:00
Reynold Xin
2905611c13
Merge pull request #851 from markhamstra/MutablePairTE
...
Removed meaningless types
2013-08-20 17:36:14 -07:00
Mark Hamstra
5eea613ec0
Removed meaningless types
2013-08-20 16:49:18 -07:00
Ali Ghodsi
f20ed14e87
Merged in from upstream to use TaskLocation instead of strings
2013-08-20 16:21:43 -07:00
Ali Ghodsi
5cd21c4195
added curly braces to make the code more consistent
2013-08-20 16:16:05 -07:00
Ali Ghodsi
db4bc55bef
indent
2013-08-20 16:16:05 -07:00
Ali Ghodsi
c0942a710f
Bug in test fixed
2013-08-20 16:16:05 -07:00
Ali Ghodsi
5db41919b5
Added a test to make sure no locality preferences are ignored
2013-08-20 16:16:05 -07:00
Ali Ghodsi
7b123b3126
Simpler code
2013-08-20 16:16:05 -07:00
Ali Ghodsi
9192c358e4
simpler code
2013-08-20 16:16:05 -07:00
Ali Ghodsi
a75a64eade
Fixed almost all of Matei's feedback
2013-08-20 16:16:05 -07:00
Ali Ghodsi
f1c853d76d
fixed Matei's comments
2013-08-20 16:16:04 -07:00
Ali Ghodsi
890ea6ba79
making CoalescedRDDPartition public
2013-08-20 16:16:04 -07:00
Ali Ghodsi
d6b6c680be
comment in the test to make it more understandable
2013-08-20 16:16:04 -07:00
Ali Ghodsi
b69e7166ba
Coalescer now uses current preferred locations for derived RDDs. Made run() in DAGScheduler thread safe and added a method to be able to ask it for preferred locations. Added a similar method that wraps the former inside SparkContext.
2013-08-20 16:16:04 -07:00
Ali Ghodsi
3b5bb8a4ae
added one test that will test a future functionality
2013-08-20 16:13:37 -07:00
Ali Ghodsi
33a0f59354
Added error messages to the tests to make failed tests less cryptic
2013-08-20 16:13:37 -07:00