Patrick Wendell
|
6be6b71c8c
|
Merge branch 'master' into ec2-updates
Conflicts:
ec2/spark_ec2.py
|
2013-08-21 15:34:31 -07:00 |
|
Jey Kottalam
|
4d737b6d32
|
Example should make sense
|
2013-08-21 15:03:37 -07:00 |
|
Jey Kottalam
|
6585f49841
|
Update build docs
|
2013-08-21 14:51:56 -07:00 |
|
Jey Kottalam
|
66e7a38a32
|
Allow build configuration to be set in conf/spark-env.sh
|
2013-08-21 14:25:53 -07:00 |
|
Mark Hamstra
|
ff6f1b0500
|
Synced sbt and maven builds
|
2013-08-21 13:50:24 -07:00 |
|
Jey Kottalam
|
31644a011c
|
Use "hadoop.version" property when specifying Hadoop YARN version too
|
2013-08-21 13:24:28 -07:00 |
|
Jey Kottalam
|
9c6f8df30f
|
Update jekyll plugin to match docs/README.md
|
2013-08-21 12:57:56 -07:00 |
|
Matei Zaharia
|
111b2741fd
|
Change default SPARK_HADOOP_VERSION in make-distribution.sh too
|
2013-08-21 11:54:10 -07:00 |
|
Reynold Xin
|
8e3ea4c7db
|
Merge branch 'master' of github.com:mesos/spark
|
2013-08-21 11:38:51 -07:00 |
|
Reynold Xin
|
af602ba9d3
|
Downgraded default build hadoop version to 1.0.4.
|
2013-08-21 11:38:24 -07:00 |
|
Matei Zaharia
|
53b1c30607
|
Update docs for Spark UI port
|
2013-08-20 22:57:11 -07:00 |
|
Patrick Wendell
|
51a1a0c602
|
Bump spark version
|
2013-08-20 22:14:52 -07:00 |
|
Reynold Xin
|
2905611c13
|
Merge pull request #851 from markhamstra/MutablePairTE
Removed meaningless types
|
2013-08-20 17:36:14 -07:00 |
|
Mark Hamstra
|
5eea613ec0
|
Removed meaningless types
|
2013-08-20 16:49:18 -07:00 |
|
Ali Ghodsi
|
f20ed14e87
|
Merged in from upstream to use TaskLocation instead of strings
|
2013-08-20 16:21:43 -07:00 |
|
Ali Ghodsi
|
5cd21c4195
|
added curly braces to make the code more consistent
|
2013-08-20 16:16:05 -07:00 |
|
Ali Ghodsi
|
db4bc55bef
|
indent
|
2013-08-20 16:16:05 -07:00 |
|
Ali Ghodsi
|
c0942a710f
|
Bug in test fixed
|
2013-08-20 16:16:05 -07:00 |
|
Ali Ghodsi
|
5db41919b5
|
Added a test to make sure no locality preferences are ignored
|
2013-08-20 16:16:05 -07:00 |
|
Ali Ghodsi
|
7b123b3126
|
Simpler code
|
2013-08-20 16:16:05 -07:00 |
|
Ali Ghodsi
|
9192c358e4
|
simpler code
|
2013-08-20 16:16:05 -07:00 |
|
Ali Ghodsi
|
a75a64eade
|
Fixed almost all of Matei's feedback
|
2013-08-20 16:16:05 -07:00 |
|
Ali Ghodsi
|
f1c853d76d
|
fixed Matei's comments
|
2013-08-20 16:16:04 -07:00 |
|
Ali Ghodsi
|
890ea6ba79
|
making CoalescedRDDPartition public
|
2013-08-20 16:16:04 -07:00 |
|
Ali Ghodsi
|
d6b6c680be
|
comment in the test to make it more understandable
|
2013-08-20 16:16:04 -07:00 |
|
Ali Ghodsi
|
b69e7166ba
|
Coalescer now uses current preferred locations for derived RDDs. Made run() in DAGScheduler thread safe and added a method to be able to ask it for preferred locations. Added a similar method that wraps the former inside SparkContext.
|
2013-08-20 16:16:04 -07:00 |
|
Ali Ghodsi
|
3b5bb8a4ae
|
added one test that will test a future functionality
|
2013-08-20 16:13:37 -07:00 |
|
Ali Ghodsi
|
33a0f59354
|
Added error messages to the tests to make failed tests less cryptic
|
2013-08-20 16:13:37 -07:00 |
|
Ali Ghodsi
|
abcefb3858
|
fixed matei's comments
|
2013-08-20 16:13:37 -07:00 |
|
Ali Ghodsi
|
35537e6341
|
Made a function object that returns the coalesced groups
|
2013-08-20 16:13:37 -07:00 |
|
Ali Ghodsi
|
339598c080
|
several of Reynold's suggestions implemented
|
2013-08-20 16:13:37 -07:00 |
|
Ali Ghodsi
|
02d6464f2f
|
space removed
|
2013-08-20 16:13:37 -07:00 |
|
Ali Ghodsi
|
4f99be1ffd
|
use count rather than foreach
|
2013-08-20 16:13:37 -07:00 |
|
Ali Ghodsi
|
f67753cdfc
|
made preferredLocation a val of the surrounding case class
|
2013-08-20 16:13:37 -07:00 |
|
Ali Ghodsi
|
f24861b60a
|
Fix bug in tests
|
2013-08-20 16:13:36 -07:00 |
|
Ali Ghodsi
|
f6e47e8b51
|
Renamed split to partition
|
2013-08-20 16:13:36 -07:00 |
|
Ali Ghodsi
|
937f72feb8
|
word wrap before 100 chars per line
|
2013-08-20 16:13:36 -07:00 |
|
Ali Ghodsi
|
c4d59910b1
|
added goals inline as comment
|
2013-08-20 16:13:36 -07:00 |
|
Ali Ghodsi
|
7a2a33e32d
|
Large scale load and locality tests for the coalesced partitions added
|
2013-08-20 16:13:36 -07:00 |
|
Ali Ghodsi
|
66edf854aa
|
Bug, should compute slack wrt parent partition size, not number of bins
|
2013-08-20 16:13:36 -07:00 |
|
Ali Ghodsi
|
1ede102ba5
|
load balancing coalescer
|
2013-08-20 16:13:36 -07:00 |
|
Patrick Wendell
|
07e5c8b695
|
Set default Hadoop version to 1
|
2013-08-20 15:49:52 -07:00 |
|
Matei Zaharia
|
aa2b89d98d
|
Merge remote-tracking branch 'jey/hadoop-agnostic'
Conflicts:
core/src/main/scala/spark/PairRDDFunctions.scala
|
2013-08-20 10:14:15 -07:00 |
|
Matei Zaharia
|
d61337f640
|
Merge pull request #844 from markhamstra/priorityRename
Renamed 'priority' to 'jobId' and assorted minor changes
|
2013-08-20 10:06:06 -07:00 |
|
Mark Hamstra
|
1630fbf838
|
changeGeneration --> changeEpoch renaming
|
2013-08-20 00:17:16 -07:00 |
|
Mark Hamstra
|
ad18410427
|
Renamed 'priority' to 'jobId' and assorted minor changes
|
2013-08-20 00:07:04 -07:00 |
|
Matei Zaharia
|
8cae72e94e
|
Merge pull request #828 from mateiz/sched-improvements
Scheduler fixes and improvements
|
2013-08-19 23:40:04 -07:00 |
|
Matei Zaharia
|
efeb142981
|
Merge pull request #849 from mateiz/web-fixes
Small fixes to web UI
|
2013-08-19 19:23:50 -07:00 |
|
Matei Zaharia
|
793a722f8e
|
Allow some wiggle room in UISuite port test and in EC2 ports
|
2013-08-19 18:51:00 -07:00 |
|
Matei Zaharia
|
abdc1f8bbb
|
Merge pull request #847 from rxin/rdd
Allow subclasses of Product2 in all key-value related classes
|
2013-08-19 18:30:56 -07:00 |
|