ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
5033 commits 6 branches 181 tags 397 MiB
Commit graph

5033 commits

This branch
This branch
All branches
Author SHA1 Message Date
Wang Jianping J 47eefd30ed add javadoc 2013-12-21 13:02:59 +08:00
Wang Jianping J 343d8977aa remove unused variable and fix a bug 2013-12-20 16:57:24 +08:00
Ankur Dave da301b57fc Merge pull request #112 from amatsukawa/scc 
Strongly connected component algorithm
 2013-12-18 20:03:29 -08:00
Akihiro Matsukawa 989f22b365 added scc tests 2013-12-18 18:22:53 -08:00
Akihiro Matsukawa b1db1b6c84 scc algorithm 2013-12-18 18:22:43 -08:00
Ankur Dave a645ef633d Merge pull request #48 from amatsukawa/add_project_to_graph 
Add mask operation on graph and filter graph primitive
 2013-12-18 16:07:52 -08:00
Akihiro Matsukawa d7ebff0272 Merge pull request #1 from ankurdave/add_project_to_graph 
Merge current master and reimplement Graph.mask using innerJoin
 2013-12-18 15:38:48 -08:00
Ankur Dave 0f137e8b75 Reimplement Graph.mask using innerJoin 2013-12-18 13:01:13 -08:00
Ankur Dave 9193a8f788 Merge remote-tracking branch 'upstream/master' into add_project_to_graph 
Conflicts:
	graph/src/main/scala/org/apache/spark/graph/Graph.scala
	graph/src/main/scala/org/apache/spark/graph/impl/GraphImpl.scala
 2013-12-18 13:00:58 -08:00
Joey 3fd2e09ffb Merge pull request #104 from jianpingjwang/master 
SVD++ demo
 2013-12-18 12:52:36 -08:00
Wang Jianping J 06581b6a96 Update Svdpp.scala 2013-12-18 06:51:37 +08:00
Wang Jianping J ba538797dd Update Svdpp.scala 2013-12-18 06:42:19 +08:00
Wang Jianping J e94fe39d0f Update Svdpp.scala 2013-12-18 06:39:28 +08:00
Joey 1b5eacbb28 Merge pull request #102 from ankurdave/clustered-edge-index 
Add clustered index on edges by source vertex
 2013-12-17 13:49:17 -08:00
Wang Jianping J 772b192910 Update AnalyticsSuite.scala 2013-12-17 19:41:04 +08:00
Wang Jianping J 9d2351f501 Update Svdpp.scala 2013-12-17 19:30:07 +08:00
Wang Jianping J aee71156f3 Test SVD++ 2013-12-17 19:20:25 +08:00
Wang Jianping J fb1d1e45f1 Update Svdpp.scala 2013-12-17 19:17:40 +08:00
Ankur Dave 3ade8be8f2 Add clustered index on edges by source vertex 
This allows efficient edge scan in mapReduceTriplets when many source
vertices are inactive. The scan method switches from edge scan to
clustered index scan when less than 80% of source vertices are active.
 2013-12-16 17:37:51 -08:00
Joey 0476c84c51 Merge pull request #100 from ankurdave/mrTriplets-active-set 
Support activeSet option in mapReduceTriplets
 2013-12-16 17:19:25 -08:00
Ankur Dave 9df565007b Merge remote-tracking branch 'upstream/master' into mrTriplets-active-set 
Conflicts:
	graph/src/main/scala/org/apache/spark/graph/impl/GraphImpl.scala
 2013-12-16 16:24:43 -08:00
Joey 5192ef3859 Merge pull request #94 from ankurdave/load-edges-columnar 
Load edges in columnar format
 2013-12-16 15:08:08 -08:00
Wang Jianping J 5eb21e6720 SVD++ demo 2013-12-16 20:01:31 +08:00
Reynold Xin ded10ce5b0 Merge pull request #103 from amplab/optimizations 
Optimizations cherry-picked from SIGMOD branches
 2013-12-15 17:25:33 -08:00
Reynold Xin bad85b051d Use murmur3 hash for open hashset. 
(cherry picked from commit 212ff6834515543163aa63a3f4f762ebe641f8ca)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
 2013-12-15 17:23:15 -08:00
Reynold Xin 62bdc44a1e Unrolled while loop in readVarLong. 
(cherry picked from commit 45ffb1ae3c0527aae50502741a3585c411875b9a)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
 2013-12-15 17:23:15 -08:00
Ankur Dave 0459747c1c Fix typo 2013-12-14 17:08:04 -08:00
Ankur Dave 8a56c1ff67 Merge pull request #84 from amatsukawa/graphlab_enhancements 
GraphLab bug fix & set start vertex
 2013-12-14 16:29:24 -08:00
Ankur Dave 1ad3d240b0 Remove commented code 2013-12-14 15:49:00 -08:00
Ankur Dave 7dbd3bf825 Remove debug printing from PregelSuite 2013-12-14 15:45:23 -08:00
Ankur Dave e9cd634d05 Merge remote-tracking branch 'upstream/master' into mrTriplets-active-set 
Updates standalone PageRank to work with the new API.
 2013-12-14 15:44:35 -08:00
Ankur Dave 0c3fc1c1b6 Avoid re-creating the view RDDs multiple times 
Previously, successive operations that support incremental view
maintenance would inadvertently recreate previous view RDDs by calling
VTableReplicated.get(), which created the RDDs anew though they were
already cached. This change memoizes the RDDs and separates the process
of shipping active-set information to an existing view.
 2013-12-14 15:28:24 -08:00
Ankur Dave 59f625b745 Revert "Add debug logging to Pregel" 
This reverts commit e62013cdd92137507a96b6a5b347a1d239209587.
 2013-12-14 15:28:24 -08:00
Ankur Dave ee5c69e481 Fix bug in VertexPartition.isActive 
This took me ~5 hours to find!
 2013-12-14 15:28:23 -08:00
Ankur Dave 4d3bba3a13 Add debug logging to Pregel 2013-12-14 15:28:23 -08:00
Ankur Dave cf6288b993 Add PregelSuite 2013-12-14 15:28:23 -08:00
Ankur Dave 9c8b6224ec Remove unused bound variable 2013-12-14 15:28:23 -08:00
Ankur Dave ef17ab58cb Add another mrTriplets test 2013-12-14 15:28:23 -08:00
Ankur Dave b40824bc2b Allow innerJoining VertexPartitions with different indexes 2013-12-14 15:28:23 -08:00
Ankur Dave 7a8952e9bb Replace skipStale with activeSetOpt in mrTriplets (fails Pregel) 2013-12-14 15:28:23 -08:00
Ankur Dave 77b92748ad Replace update with innerJoin (has a bug) 
There is a conflict between vertices that didn't change so are not moved
but still need to run, and vertices that were deleted by the innerJoin
so should not run.
 2013-12-14 15:28:23 -08:00
Ankur Dave d161caa6eb Expose srcStale and dstStale 2013-12-14 15:28:23 -08:00
Ankur Dave 4bbae5c39b Remove printlns from test 2013-12-14 15:28:23 -08:00
Ankur Dave b2f595f552 Test more graph ops and skipStale 2013-12-14 15:28:22 -08:00
Ankur Dave 45ea674e21 Handle diffing VertexPartitions with different indexes 2013-12-14 15:28:22 -08:00
Ankur Dave 8f4b8e9b95 Reuse previous localVidMap if available 2013-12-14 15:28:22 -08:00
Ankur Dave 5e20cbaf66 Define localVidMap once per VTableReplicated 2013-12-14 15:28:22 -08:00
Ankur Dave dc72147900 Fix bug in interaction of incr. view maint., skipStale, and join rewrite 2013-12-14 15:28:22 -08:00
Ankur Dave d076fe5c9e Finish up 49f7fc5ed2edd42b118182438d84ed7d4e7d521f 2013-12-14 15:28:22 -08:00
Ankur Dave 4f4f0add24 Fix bug in mapVertices and outerJoinVertices 2013-12-14 15:28:22 -08:00