Joseph E. Gonzalez
41b3122120
Strating to improve README.
2013-10-29 20:57:55 -07:00
Joseph E. Gonzalez
38ec0baf5c
fixing a typo in the VertexSetRDD docs
2013-10-29 16:27:55 -07:00
Joseph E. Gonzalez
d8c8256e52
merging upstream changes
2013-10-29 16:23:26 -07:00
Ankur Dave
098768e0b9
Merge pull request #37 from jegonzal/AnalyticsCleanup
...
Updated Connected Components and Pregel Docs
2013-10-29 15:08:36 -07:00
Joseph E. Gonzalez
08c7b040d6
Documented the VertexSetRDD
2013-10-29 15:03:13 -07:00
Joseph E. Gonzalez
ede329336d
Fixing a scaladoc bug in graph generators.
2013-10-29 14:50:12 -07:00
Joseph E. Gonzalez
15958ca65a
Reindenting documentation.
2013-10-29 14:01:24 -07:00
Joseph E. Gonzalez
d316cad9b1
Documented Graph.appy functions.
2013-10-29 13:58:04 -07:00
Joseph E. Gonzalez
19da8820fc
Minor modifications to documentation.
2013-10-29 11:06:06 -07:00
Joseph E. Gonzalez
77626d1507
Adding collect neighbors and documenting GraphOps.
2013-10-29 11:05:42 -07:00
Joseph E. Gonzalez
942de98433
Making suggested changes.
2013-10-29 10:19:49 -07:00
Joseph E. Gonzalez
d6a902f309
Finished updating connected components to used Pregel like abstraction and created a series of tests in the AnalyticsSuite.
2013-10-28 11:52:26 -07:00
Joseph E. Gonzalez
a2287ae138
Implementing connected components on top of pregel like abstraction.
2013-10-27 10:42:11 -07:00
Joseph E. Gonzalez
6a0fbc0374
Updating the GraphLab API to match the changes made to the Pregel API.
2013-10-26 15:44:19 -07:00
Joseph E. Gonzalez
08024c938c
Adding more documentation to the Pregel API as well as additional functionality including the ability to specify the edge direction along which messages are computed.
2013-10-26 15:42:51 -07:00
Joseph E. Gonzalez
00e73833cc
Fixing a bug in reverse edge direction.
2013-10-26 15:10:30 -07:00
Reynold Xin
6f82c42690
Merge pull request #34 from jegonzal/AnalyticsCleanup
...
Analytics Cleanup
2013-10-24 11:09:46 -07:00
Joseph E. Gonzalez
9d1e5946fe
Merge branch 'master' of https://github.com/amplab/graphx into AnalyticsCleanup
2013-10-23 00:26:06 -07:00
Joseph E. Gonzalez
c30624dcbb
Adding dynamic pregel, fixing bugs in PageRank, and adding basic analytics unit tests.
2013-10-23 00:25:45 -07:00
Joseph E. Gonzalez
0bd92ed8d0
Fixing a bug in pregel where the initial vertex-program results are lost.
2013-10-22 19:10:51 -07:00
Dan Crankshaw
49d5cdac33
Merge pull request #30 from jegonzal/VertexSetRDD_Tests
...
Testing and Documenting VertexSetRDD
2013-10-22 15:38:02 -07:00
Joseph E. Gonzalez
be8269af07
Merge branch 'VertexSetRDD_Tests' into AnalyticsCleanup
2013-10-22 15:03:49 -07:00
Joseph E. Gonzalez
e3eb03d5b5
Starting analytics test suite.
2013-10-22 15:03:16 -07:00
Joseph E. Gonzalez
ba5c75692a
Updating analytics to reflect changes in the pregel interface and moving degree information into the edge attribute.
2013-10-22 15:03:00 -07:00
Joseph E. Gonzalez
46b195253e
Adding some additional graph generators to support unit testing of the analytics package.
2013-10-22 15:01:49 -07:00
Joseph E. Gonzalez
14a3329a11
Changing the Pregel interface slightly to better support type inference.
2013-10-22 15:01:20 -07:00
Joseph E. Gonzalez
ebdbedc3e9
Documenting VertexSetRDD and added some testing code for VertexSetRDD
2013-10-19 01:26:08 -07:00
Joseph E. Gonzalez
dbc8c9868a
Fixing bug in VertexSetRDD that breaks Graph tests.
2013-10-18 23:44:06 -07:00
Reynold Xin
9cf43cfeb7
Merge pull request #28 from jegonzal/VertexSetRDD
...
Refactoring IndexedRDD to VertexSetRDD.
2013-10-18 22:07:21 -07:00
Reynold Xin
f888a5b051
Merge pull request #29 from ankurdave/unit-tests
...
Unit tests for Graph and GraphOps
2013-10-18 22:06:58 -07:00
Ankur Dave
2d3603930e
Add a unit test for GraphOps.joinVertices
2013-10-18 19:46:13 -07:00
Ankur Dave
d15db10831
Add a unit test for Graph.mapEdges
2013-10-18 19:46:13 -07:00
Ankur Dave
d429f015c0
Update GraphSuite aggregateNeighbors test
2013-10-18 19:46:13 -07:00
Joseph E. Gonzalez
5d01ebca3c
Specializing IndexedRDD as VertexSetRDD.
...
1) This allows the index map to be optimized for Vids
2) This makes the code more readable
2) The Graph API can now return VertexSetRDDs from operations that produce results for vertices
2013-10-18 19:03:59 -07:00
Ankur Dave
0794bd7bc5
Merge pull request #27 from jegonzal/removed_indexedrdd_from_core
...
Removing IndexedRDD changes for spark/core
2013-10-18 18:59:58 -07:00
Joseph E. Gonzalez
bb58aa5330
Added some stub code to address the case where a vertex could occur multiple times in the vertex table or where a vertex in the edge list may not appear in the vertex table.
...
Moving IndexedRDD into the graphx source tree and removing dependencies in /core.
2013-10-18 18:15:32 -07:00
Joseph E. Gonzalez
fc5af50a2f
Merge branch 'master' of https://github.com/amplab/graphx
2013-10-18 18:15:17 -07:00
Ankur Dave
36a902e52d
Revert accidental removal of code in 3a40a5e
2013-10-18 16:19:40 -07:00
Ankur Dave
971f824014
Revert unnecessary changes to core
...
While benchmarking, we accidentally committed some unnecessary changes
to core such as adding logging. These changes make it more difficult to
merge from Spark upstream, so this commit reverts them.
2013-10-18 16:07:38 -07:00
Dan Crankshaw
8bd5f89662
Merge branch 'indexedrdd_graphx' of github.com:amplab/graphx into indexedrdd_graphx
2013-10-18 15:11:28 -07:00
Dan Crankshaw
3a40a5eb30
Added some documentation.
2013-10-18 15:11:21 -07:00
Joseph E. Gonzalez
1856b37e9d
Merge branch 'master' of https://github.com/apache/incubator-spark into indexedrdd_graphx
2013-10-18 12:21:19 -07:00
Joseph E. Gonzalez
e028079b0f
Merging with spark upstream changes.
2013-10-18 12:02:14 -07:00
Joseph E. Gonzalez
3f3d28c73f
Switching from Seq to IndexedSeq
2013-10-17 19:55:36 -07:00
Joseph E. Gonzalez
9a03c5fe28
This commit accomplishes three goals:
...
1) Further simplification of the IndexedRDD operations (eliminating some)
2) Aggressive reuse of HashMaps
3) Pipelining join operations within indexedrdd
2013-10-17 19:01:48 -07:00
Joey
099977fd1b
Merge pull request #26 from ankurdave/split-vTableReplicated
...
Great work!
2013-10-17 14:17:08 -07:00
Matei Zaharia
fc26e5b832
Merge pull request #69 from KarthikTunga/master
...
Fix for issue SPARK-627. Implementing --config argument in the scripts.
This code fix is for issue SPARK-627. I added code to consider --config arguments in the scripts. In case the <conf-dir> is not a directory the scripts exit. I removed the --hosts argument. It can be achieved by giving a different config directory. Let me know if an explicit --hosts argument is required.
2013-10-17 13:21:07 -07:00
Ankur Dave
bf19aac2b7
Use ArrayBuilder instead of ArrayBuffer
...
ArrayBuilder is specialized for holding primitive VD types.
2013-10-17 13:19:00 -07:00
Matei Zaharia
cf64f63f8a
Merge pull request #67 from kayousterhout/remove_tsl
...
Removed TaskSchedulerListener interface.
The interface was used only by the DAG scheduler (so it wasn't necessary
to define the additional interface), and the naming makes it very
confusing when reading the code (because "listener" was used
to describe the DAG scheduler, rather than SparkListeners, which
implement a nearly-identical interface but serve a different
function).
@mateiz - is there a reason for this interface that I'm missing?
2013-10-17 11:12:28 -07:00
Ankur Dave
2282d27cf1
Cache msgsByPartition
2013-10-16 23:56:15 -07:00