ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
5030 commits 6 branches 181 tags 397 MiB
Commit graph

80 commits

Author SHA1 Message Date
Akihiro Matsukawa 989f22b365 added scc tests 2013-12-18 18:22:53 -08:00
Ankur Dave 0f137e8b75 Reimplement Graph.mask using innerJoin 2013-12-18 13:01:13 -08:00
Ankur Dave 9193a8f788 Merge remote-tracking branch 'upstream/master' into add_project_to_graph 
Conflicts:
	graph/src/main/scala/org/apache/spark/graph/Graph.scala
	graph/src/main/scala/org/apache/spark/graph/impl/GraphImpl.scala
 2013-12-18 13:00:58 -08:00
Joey 3fd2e09ffb Merge pull request #104 from jianpingjwang/master 
SVD++ demo
 2013-12-18 12:52:36 -08:00
Wang Jianping J 772b192910 Update AnalyticsSuite.scala 2013-12-17 19:41:04 +08:00
Wang Jianping J aee71156f3 Test SVD++ 2013-12-17 19:20:25 +08:00
Ankur Dave 3ade8be8f2 Add clustered index on edges by source vertex 
This allows efficient edge scan in mapReduceTriplets when many source
vertices are inactive. The scan method switches from edge scan to
clustered index scan when less than 80% of source vertices are active.
 2013-12-16 17:37:51 -08:00
Ankur Dave 9df565007b Merge remote-tracking branch 'upstream/master' into mrTriplets-active-set 
Conflicts:
	graph/src/main/scala/org/apache/spark/graph/impl/GraphImpl.scala
 2013-12-16 16:24:43 -08:00
Ankur Dave 7dbd3bf825 Remove debug printing from PregelSuite 2013-12-14 15:45:23 -08:00
Ankur Dave e9cd634d05 Merge remote-tracking branch 'upstream/master' into mrTriplets-active-set 
Updates standalone PageRank to work with the new API.
 2013-12-14 15:44:35 -08:00
Ankur Dave cf6288b993 Add PregelSuite 2013-12-14 15:28:23 -08:00
Ankur Dave ef17ab58cb Add another mrTriplets test 2013-12-14 15:28:23 -08:00
Ankur Dave 7a8952e9bb Replace skipStale with activeSetOpt in mrTriplets (fails Pregel) 2013-12-14 15:28:23 -08:00
Ankur Dave 4bbae5c39b Remove printlns from test 2013-12-14 15:28:23 -08:00
Ankur Dave b2f595f552 Test more graph ops and skipStale 2013-12-14 15:28:22 -08:00
Ankur Dave 87f2909561 Incremental view maintenance for all graph ops 
All GraphImpl operations now support incremental maintenance of the
replicated vertex view by reusing vTableReplicated whenever
possible. This has two consequences:

- This is not confined to the deltaJoinVertices operator anymore,
  so we rename it to the more intuitive "updateVertices" instead.

- This enables mrTriplets to support skipStale.
 2013-12-14 15:28:22 -08:00
Ankur Dave d00cc8092b Fix argument bug and closure capture 2013-12-14 15:03:57 -08:00
Ankur Dave a8c7ebf0ed Don't partition edges by default; refactor 
Instead, expose Graph.partitionBy(PartitionStrategy).
 2013-12-14 15:03:57 -08:00
Ankur Dave a0fb477726 Test standalone PageRank 2013-12-12 15:42:55 -08:00
Reynold Xin 41721b1494 Fixed a bug in VTableReplicated that we only process the first block. 2013-12-06 00:51:12 -08:00
Reynold Xin 3b0ee53eda Minor update. 2013-12-05 23:30:56 -08:00
Reynold Xin 15168d6c4d Fixed a bug in VTableReplicated that we are always broadcasting all the vertices. 2013-12-05 23:25:53 -08:00
Reynold Xin a6075ba11f Merge branch 'pregel-replicate-changed' of github.com:ankurdave/graphx into pregel-replicate-changed 2013-12-05 22:35:16 -08:00
Ankur Dave 67bfa7fd65 Test prevVTableReplicated code path 2013-12-05 19:16:00 -08:00
Reynold Xin 920282c36a Merge branch 'pregel-replicate-changed' of github.com:ankurdave/graphx into pregel-replicate-changed 2013-12-05 18:57:36 -08:00
Reynold Xin 4f80dd22bd Fixed a bug that variable encoding doesn't work for ints that use all 64 bits. 2013-12-05 16:19:37 -08:00
Ankur Dave 4d4c1c931e Test Graph.deltaJoinVertices and VertexRDD.diff 2013-12-05 16:15:56 -08:00
Ankur Dave 2e583d2de4 Declare Vids explicitly to avoid ClassCastException 2013-12-04 17:34:14 -08:00
Ankur Dave 92e96f727e Fix compile errors in GraphSuite and SerializerSuite 2013-12-04 17:29:52 -08:00
Akihiro Matsukawa c888b611da add mask operation to Graph 2013-12-04 16:48:32 -08:00
Reynold Xin 55edbb4209 Created an algorithms package and put all algorithms there. 2013-12-01 20:17:26 -08:00
Reynold Xin 6eeadb667d Created EdgeRDD. 2013-11-30 16:53:54 -08:00
Reynold Xin 34ee81415e Merged Ankur's pull request #80 and fixed subgraph. 2013-11-30 15:10:30 -08:00
Reynold Xin 8e790b7f7a Merge branch 'subgraph-test' of github.com:ankurdave/graphx into clean1 
Conflicts:
	graph/src/main/scala/org/apache/spark/graph/impl/VertexPartition.scala
 2013-11-30 14:48:43 -08:00
Reynold Xin 229022891f Made all VertexPartition internal data structures private. 2013-11-30 14:45:56 -08:00
Reynold Xin 4d3d68b8fb Minor update to tests. 2013-11-30 14:15:47 -08:00
Ankur Dave eed3195038 Fix VertexSetRDD test by enabling index reuse 2013-11-30 13:50:37 -08:00
Reynold Xin 10c0f9b0bb Added a log4j properties file for graphx unit tests. 2013-11-30 13:18:43 -08:00
Reynold Xin 95e83af209 More, bigger cleaning for better encapsulation of VertexSetRDD and VertexPartition. This is work in progress as stuff doesn't really run. 2013-11-27 00:30:26 -08:00
Ankur Dave 9e896be375 Test edge filtering in subgraph (test fails) 2013-11-26 15:58:55 -08:00
Ankur Dave 137294e2ab Test GraphImpl.subgraph and fix bug 2013-11-26 15:32:47 -08:00
Joseph E. Gonzalez 18700b6e74 Switching mapReduceTriplets mapFunction to return iterator instead of array to allow optimizations of the returned object. 2013-11-19 21:36:15 -08:00
Joseph E. Gonzalez 983810ad69 Now with style. Addressing most of Reynolds comments. 2013-11-19 21:35:03 -08:00
Joseph E. Gonzalez 2093a17ff3 Adding triangle count code 2013-11-19 21:35:03 -08:00
Joseph E. Gonzalez 8719ba83c8 Modifying graph loaders to create initial vertex sets more efficiently and load undirected graphs. 2013-11-19 21:35:02 -08:00
Joseph E. Gonzalez 288ae310e7 adding test for collectNeighborIds 2013-11-19 21:03:00 -08:00
Ankur Dave 62a2a71c37 Merge pull request #65 from amplab/varenc 
Use variable encoding for ints, longs, and doubles in the specialized serializers.
 2013-11-15 13:12:07 -08:00
Joseph E. Gonzalez 266eb01ce8 Addressing issue in Graph creation where a graph created with a vertex set that does not span all of the vertices in the edges will crash on triplet construction. 2013-11-13 10:45:25 -08:00
Reynold Xin 882d069189 Fixed the bug in variable encoding for longs. 2013-11-12 18:50:03 -08:00
Ankur Dave aa24b0bbe8 Add test for mapReduceTriplets in GraphSuite 2013-11-10 14:24:38 -08:00