ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
4973 commits 6 branches 181 tags 397 MiB
Commit graph

4973 commits

This branch
This branch
All branches
Author SHA1 Message Date
Reynold Xin 62bdc44a1e Unrolled while loop in readVarLong. 
(cherry picked from commit 45ffb1ae3c0527aae50502741a3585c411875b9a)
Signed-off-by: Ankur Dave <ankurdave@gmail.com>
 2013-12-15 17:23:15 -08:00
Reynold Xin 9bf192b01c Merge pull request #91 from amplab/standalone-pagerank 
Standalone PageRank
 2013-12-14 12:52:18 -08:00
Reynold Xin 840af5e8e4 Merge pull request #99 from ankurdave/only-dynamic-pregel 
Remove static Pregel; take maxIterations in dynamic Pregel
 2013-12-14 12:51:51 -08:00
Ankur Dave 5b0d6f0ad5 Remove static Pregel; take maxIters in dynamic Pregel 2013-12-12 18:03:19 -08:00
Ankur Dave 3f69cdc81b Use standalone PageRank in Analytics 2013-12-12 15:43:12 -08:00
Ankur Dave a0fb477726 Test standalone PageRank 2013-12-12 15:42:55 -08:00
Ankur Dave e8ba51d644 Add standalone PageRank using only GraphX operators 2013-12-12 15:42:27 -08:00
Reynold Xin ce6ca4ea61 Merge pull request #97 from dcrankshaw/fix-rddtop 
Added BoundedPriorityQueue kryo registrator. Fixes top issue.
 2013-12-11 22:30:54 -08:00
Dan Crankshaw 12483d4ae6 Added BoundedPriorityQueue kryo registrator. Fixes top issue. 2013-12-11 20:56:21 -08:00
Reynold Xin 3abfbfb104 Merge pull request #92 from ankurdave/rdd-names 
Set RDD names for easy debugging
 2013-12-07 11:24:19 -08:00
Ankur Dave 84d0e1a334 Set RDD names for easy debugging 2013-12-07 04:05:45 -08:00
Ankur Dave 31e8a14e17 Merge pull request #90 from amplab/pregel-replicate-changed 
Replicate only changed vertices
 2013-12-06 21:49:55 -08:00
Reynold Xin 41721b1494 Fixed a bug in VTableReplicated that we only process the first block. 2013-12-06 00:51:12 -08:00
Reynold Xin 3b0ee53eda Minor update. 2013-12-05 23:30:56 -08:00
Reynold Xin 15168d6c4d Fixed a bug in VTableReplicated that we are always broadcasting all the vertices. 2013-12-05 23:25:53 -08:00
Reynold Xin a6075ba11f Merge branch 'pregel-replicate-changed' of github.com:ankurdave/graphx into pregel-replicate-changed 2013-12-05 22:35:16 -08:00
Ankur Dave b707861ba0 Simplify GraphImpl.deltaJoinVertices 2013-12-05 20:01:32 -08:00
Ankur Dave 67bfa7fd65 Test prevVTableReplicated code path 2013-12-05 19:16:00 -08:00
Reynold Xin 920282c36a Merge branch 'pregel-replicate-changed' of github.com:ankurdave/graphx into pregel-replicate-changed 2013-12-05 18:57:36 -08:00
Ankur Dave 39b0256668 Use mask for dynamic Pregel 2013-12-05 18:55:59 -08:00
Ankur Dave 7457abe282 Re-roll last iteration of static Pregel loop 2013-12-05 16:59:49 -08:00
Ankur Dave 1c8500efc0 Merge pull request #88 from amplab/varenc 
Fixed a bug that variable encoding doesn't work for ints that use all 64 bits.
 2013-12-05 16:25:44 -08:00
Reynold Xin 4f80dd22bd Fixed a bug that variable encoding doesn't work for ints that use all 64 bits. 2013-12-05 16:19:37 -08:00
Ankur Dave 3d57be6151 Revert changes to project/SparkBuild.scala 
Changes were made in 3fc4534d19
 2013-12-05 16:19:26 -08:00
Ankur Dave 4d4c1c931e Test Graph.deltaJoinVertices and VertexRDD.diff 2013-12-05 16:15:56 -08:00
Ankur Dave 15374fed97 In static Pregel, replicate only changed vertices 2013-12-05 15:55:33 -08:00
Reynold Xin 3fc4534d19 wip delta join. 2013-12-05 14:55:26 -08:00
Ankur Dave e0bcaa0942 Merge pull request #86 from ankurdave/vid-varenc 
Finish work on #85
 2013-12-05 12:37:02 -08:00
Ankur Dave 6a7b396e5d Finish work on #85 2013-12-05 12:35:03 -08:00
Reynold Xin 3e96b9ab1e Merge pull request #85 from ankurdave/vid-varenc 
Always write Vids using variable encoding
 2013-12-05 12:07:36 -08:00
Ankur Dave a3bb98b88a Always write Vids using variable encoding 
Also, autoformat Serializers.scala.
 2013-12-05 12:06:07 -08:00
Joey e0347ba6c7 Merge pull request #83 from ankurdave/fix-tests 
Fix compile errors in GraphSuite and SerializerSuite
 2013-12-04 17:38:06 -08:00
Ankur Dave 2e583d2de4 Declare Vids explicitly to avoid ClassCastException 2013-12-04 17:34:14 -08:00
Ankur Dave 92e96f727e Fix compile errors in GraphSuite and SerializerSuite 2013-12-04 17:29:52 -08:00
Reynold Xin cbd3b7571f Merge pull request #81 from amplab/clean1 
Codebase refactoring
 2013-12-04 15:35:26 -08:00
Reynold Xin 8701cb55e6 Use specialized shuffler for aggregation. 2013-12-01 21:55:50 -08:00
Reynold Xin 55edbb4209 Created an algorithms package and put all algorithms there. 2013-12-01 20:17:26 -08:00
Reynold Xin e36fe55a03 Memoize preferred locations in ZippedPartitionsBaseRDD so preferred location computation doesn't lead to exponential explosion. 2013-11-30 18:07:36 -08:00
Reynold Xin 583a389e3f Removed PartitionStrategy from GraphImpl. 2013-11-30 17:00:54 -08:00
Reynold Xin 6eeadb667d Created EdgeRDD. 2013-11-30 16:53:54 -08:00
Reynold Xin 34ee81415e Merged Ankur's pull request #80 and fixed subgraph. 2013-11-30 15:10:30 -08:00
Reynold Xin 8e790b7f7a Merge branch 'subgraph-test' of github.com:ankurdave/graphx into clean1 
Conflicts:
	graph/src/main/scala/org/apache/spark/graph/impl/VertexPartition.scala
 2013-11-30 14:48:43 -08:00
Reynold Xin 229022891f Made all VertexPartition internal data structures private. 2013-11-30 14:45:56 -08:00
Reynold Xin b30e0ae035 Added an optimized count to VertexSetRDD. 2013-11-30 14:24:18 -08:00
Reynold Xin 689f757f7a Merge branch 'clean1' of github.com:amplab/graphx into clean1 2013-11-30 14:16:06 -08:00
Reynold Xin 4d3d68b8fb Minor update to tests. 2013-11-30 14:15:47 -08:00
Ankur Dave 3292cb0f9c Revert "Fix join error by caching vTable in mapReduceTriplets" 
This reverts commit dee1318d3d, which is
unnecessary due to 7528e6d5f1.
 2013-11-30 14:05:32 -08:00
Reynold Xin e72bd91590 Merge branch 'clean1' of github.com:amplab/graphx into clean1 2013-11-30 14:04:45 -08:00
Reynold Xin 7528e6d5f1 Enable joining arbitrary VertexPartitions (with different indexes). 2013-11-30 14:04:16 -08:00
Ankur Dave eed3195038 Fix VertexSetRDD test by enabling index reuse 2013-11-30 13:50:37 -08:00