Commit graph

114 commits

Author SHA1 Message Date
Ankur Dave 6d707f6b63 Remove ShortestPath for now 2011-10-09 16:19:34 -07:00
Ankur Dave 0028caf3a4 Simplify and genericize type parameters in Bagel 2011-10-09 15:58:39 -07:00
Ismael Juma 0fba22b3d2 Fix issue #65: Change @serializable to extends Serializable in 2.9 branch
Note that we use scala.Serializable introduced in Scala 2.9 instead of
java.io.Serializable. Also, case classes inherit from scala.Serializable by
default.
2011-08-02 10:16:33 +01:00
Matei Zaharia 969644df8e Cleaned up a few issues to do with default parallelism levels. Also
renamed HadoopFileWriter to HadoopWriter (since it's not only for files)
and fixed a bug for lookup().
2011-07-14 12:40:56 -04:00
Matei Zaharia 4db50e26c7 Fixed unit tests by making them clean up the SparkContext after use and
thus clean up the various singletons (RDDCache, MapOutputTracker, etc).
This isn't perfect yet (ideally we shouldn't use singleton objects at
all) but we can fix that later.
2011-05-13 12:03:58 -07:00
Ankur Dave f40a0898a7 Rename bagel to spark.bagel and Pregel to Bagel 2011-05-09 15:23:21 -07:00
Ankur Dave c1104058c6 Move shortest path and PageRank to bagel.examples 2011-05-03 18:53:58 -07:00
Ankur Dave 563c5e717c Refactor and add aggregator support
Refactored out the agg() and comp() methods from Pregel.run.

Defined an implicit conversion to allow applications that don't use
aggregators to avoid including a null argument for the result of the
aggregator in the compute function.
2011-05-03 15:40:45 -07:00
Ankur Dave c18fa3ebc6 Package combiner functions into a trait 2011-05-03 15:40:41 -07:00
Ankur Dave 1c8ca0ebe1 Add Bagel test suite
Note: This test suite currently fails for the same reason that the
Spark Core test suite fails: Spark currently seems to have a bug where
any test after the first one fails.
2011-05-03 15:40:31 -07:00
Ankur Dave c5b3ea755f Clean up Bagel source and interface 2011-05-03 15:40:01 -07:00
Ankur Dave 19122af787 Update ShortestPath to work with controllable partitioning 2011-05-03 15:39:39 -07:00
Ankur Dave 62ef620354 Clean up Pregel.run, add logging 2011-05-03 15:38:01 -07:00
Ankur Dave c0736f6f68 Add Bagel, an implementation of Pregel on Spark 2011-05-03 15:37:08 -07:00