Matei Zaharia
|
600e99728d
|
Fix a bug where an input path was added to a Hadoop job configuration twice
|
2012-07-23 16:16:19 -07:00 |
|
Josh Rosen
|
042dcbde33
|
Add type annotations to Java API methods.
Add missing Scala Map to java.util.Map conversions.
|
2012-07-22 17:35:29 -07:00 |
|
Josh Rosen
|
e23938c3be
|
Use mapValues() in JavaPairRDD.cogroupResultToJava().
|
2012-07-22 15:10:01 -07:00 |
|
Josh Rosen
|
460da878fc
|
Improve Java API examples
- Replace JavaLR example with JavaHdfsLR example.
- Use anonymous classes in JavaWordCount; add options.
- Remove @Override annotations.
|
2012-07-22 14:40:39 -07:00 |
|
Josh Rosen
|
01dce3f569
|
Add Java API
Add distinct() method to RDD.
Fix bug in DoubleRDDFunctions.
|
2012-07-18 17:34:29 -07:00 |
|
Matei Zaharia
|
628bb5ca7f
|
Allow null keys in Spark's reduce and group by
|
2012-07-12 18:36:02 -07:00 |
|
Matei Zaharia
|
e2a67a8024
|
Fixes to coarse-grained Mesos scheduler in dealing with failed nodes
|
2012-07-12 18:21:52 -07:00 |
|
Matei Zaharia
|
be622cf867
|
Formatting
|
2012-07-11 17:31:44 -07:00 |
|
Matei Zaharia
|
e8ae77df24
|
Added more methods for loading/saving with new Hadoop API
|
2012-07-11 17:31:33 -07:00 |
|
Matei Zaharia
|
0a47284003
|
More work to allow Spark to run on the standalone deploy cluster.
|
2012-07-08 14:00:04 -07:00 |
|
Matei Zaharia
|
1aa63f775b
|
Added back coarse-grained Mesos scheduler based on StandaloneScheduler.
|
2012-07-08 10:52:13 -07:00 |
|
Matei Zaharia
|
c5cc10cda3
|
More work on standalone scheduler
|
2012-07-06 20:17:44 -07:00 |
|
Matei Zaharia
|
909b325243
|
Further refactoring, and start of a standalone scheduler backend
|
2012-07-06 17:56:44 -07:00 |
|
Matei Zaharia
|
4e2fe0bdaf
|
Miscellaneous bug fixes
|
2012-07-06 16:33:40 -07:00 |
|
Matei Zaharia
|
e72afdb817
|
Some refactoring to make cluster scheduler pluggable.
|
2012-07-06 15:23:26 -07:00 |
|
Matei Zaharia
|
5d1a887bed
|
Further updates to run processes on cluster.
|
2012-07-01 17:13:31 -07:00 |
|
Matei Zaharia
|
51c46eaca0
|
More work on standalone deploy system.
|
2012-07-01 01:05:59 -07:00 |
|
Matei Zaharia
|
a6eb9fda61
|
Detect connection and disconnection of slaves
|
2012-06-30 17:46:56 -07:00 |
|
Matei Zaharia
|
408b5a1332
|
More work on deploy code (adding Worker class)
|
2012-06-30 16:45:57 -07:00 |
|
Matei Zaharia
|
2fb6e7d71e
|
Initial framework to get a master and web UI up.
|
2012-06-30 14:45:55 -07:00 |
|
Matei Zaharia
|
c53670b9bf
|
Various code style fixes, mostly from IntelliJ IDEA
|
2012-06-29 18:47:12 -07:00 |
|
rrmckinley
|
697b0bee2c
|
Scalacheck groupId has changed https://github.com/rickynils/scalacheck/issues/24. Necessary to build with scalaVersion 2.9.2. Works with 2.9.1 too.
|
2012-06-29 16:42:05 -07:00 |
|
Matei Zaharia
|
8a5bd4e765
|
Increase the default wait time for EC2 clusters to 2 minutes.
|
2012-06-29 16:25:40 -07:00 |
|
Matei Zaharia
|
c6be4ffbf9
|
Fixes to CoarseMesosScheduler
|
2012-06-29 16:18:51 -07:00 |
|
Matei Zaharia
|
3a58efa5a5
|
Allow binding to a free port and change Akka logging to use SLF4J. Also
fixes various bugs in the previous code when running on Mesos.
|
2012-06-29 16:02:21 -07:00 |
|
Matei Zaharia
|
14bfad1183
|
Ignore some IDE files
|
2012-06-28 23:52:33 -07:00 |
|
Matei Zaharia
|
3920189932
|
Upgraded to Akka 2 and fixed test execution (which was still parallel
across projects).
|
2012-06-28 23:51:28 -07:00 |
|
Tathagata Das
|
ede615d719
|
Fixed issues duplicate class issues in sbt assembly.
|
2012-06-22 15:03:09 -07:00 |
|
root
|
6ad3e1f1b4
|
Various fixes when running on Mesos
|
2012-06-20 06:48:26 +00:00 |
|
Tathagata Das
|
e896a505e2
|
Added testcase for ByteBufferInputStream bugs.
|
2012-06-17 16:11:12 -07:00 |
|
Matei Zaharia
|
3e0396c953
|
Update SBT and SBT-Eclipse version
|
2012-06-17 14:37:18 -07:00 |
|
Tathagata Das
|
40536e3668
|
Fixed nasty corner case bug in ByteBufferInputStream. Could not add a test case for this as I could not figure out how to deterministically reproduce the bug in a short testcase.
|
2012-06-17 13:28:41 -07:00 |
|
Matei Zaharia
|
2893b30550
|
Various fixes to get unit tests running. In particular, shut down
ConnectionManager and DAGScheduler properly, plus a fix to
LocalScheduler that was not merged in from 0.5 and was actually caught
by one of the tests.
|
2012-06-17 00:28:45 -07:00 |
|
Matei Zaharia
|
08579ffa11
|
Update version number for dev branch
|
2012-06-15 23:55:43 -07:00 |
|
Matei Zaharia
|
b3eeac55b8
|
Fixed HttpBroadcast to work with this branch's Serializer.
|
2012-06-15 23:54:38 -07:00 |
|
Matei Zaharia
|
f58da6164e
|
Merge branch 'master' into dev
|
2012-06-15 23:47:11 -07:00 |
|
Matei Zaharia
|
b5cf47cda3
|
Merge branch 'dev' of github.com:mesos/spark into dev
|
2012-06-15 23:36:30 -07:00 |
|
Matei Zaharia
|
4449eb9783
|
Changed version in master branch to 0.5.1-SNAPSHOT for further
development.
|
2012-06-13 22:26:14 -04:00 |
|
Tathagata Das
|
5f54bdf98b
|
Added shutdown for akka to SparkContext.stop(). Helps a little, but many testsuites still fail.
|
2012-06-13 20:49:00 -04:00 |
|
Tathagata Das
|
c6156da9e2
|
Multiple bug fixes to pass the testsuites ShuffleSuite and BlockManagerSuite.
|
2012-06-13 16:26:49 -04:00 |
|
Matei Zaharia
|
0472cf8e0d
|
Update version in SBT
|
2012-06-12 14:30:49 -04:00 |
|
Matei Zaharia
|
4971e0f547
|
Updated version number to 0.5.0
|
2012-06-12 13:41:57 -04:00 |
|
Matei Zaharia
|
08c50ad1fc
|
Added script for launching Spark on EC2 from Mesos, to make it easier
for new users to get up and running on EC2.
|
2012-06-10 23:06:15 -07:00 |
|
Matei Zaharia
|
879bc0bece
|
Merge branch 'master' into mesos-0.9
|
2012-06-09 16:24:16 -07:00 |
|
Matei Zaharia
|
4b05798c06
|
Further bug fix to HttpBroadcast
|
2012-06-09 16:24:03 -07:00 |
|
Matei Zaharia
|
587a16a7ef
|
Merge branch 'master' into mesos-0.9
|
2012-06-09 16:17:07 -07:00 |
|
Matei Zaharia
|
8ed662862e
|
Bug fix to HttpBroadcast
|
2012-06-09 16:16:55 -07:00 |
|
Matei Zaharia
|
2fd9f994ae
|
Merge branch 'master' into mesos-0.9
|
2012-06-09 15:58:35 -07:00 |
|
Matei Zaharia
|
e75b1b5cb4
|
Change the default broadcast implementation to a simple HTTP-based
broadcast. Fixes #139.
|
2012-06-09 15:58:07 -07:00 |
|
Matei Zaharia
|
a96558caa3
|
Performance improvements to shuffle operations: in particular, preserve
RDD partitioning in more cases where it's possible, and use iterators
instead of materializing collections when doing joins.
|
2012-06-09 14:44:18 -07:00 |
|