ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
5072 commits 6 branches 181 tags 397 MiB
Commit graph

5072 commits

This branch
This branch
All branches
Author SHA1 Message Date
Adam Novak fa8ce3fdd7 Changing org.apache.spark.util.collection.PrimitiveKeyOpenHashMap to have a real no-argument constructor, instead of a one-argument constructor with a default value. The lack of a real no-argument constructor was causing "sbt/sbt publish-local" to fail thusly: 
```
[error] /pod/home/anovak/build/graphx/core/src/main/scala/org/apache/spark/storage/ShuffleBlockManager.scala:172: not enough arguments for constructor PrimitiveKeyOpenHashMap: (initialCapacity: Int)(implicit evidence$3: ClassManifest[Int], implicit evidence$4: ClassManifest[Int])org.apache.spark.util.collection.PrimitiveKeyOpenHashMap[Int,Int]
[error]     private val mapIdToIndex = new PrimitiveKeyOpenHashMap[Int, Int]()
[error]                                ^
[info] No documentation generated with unsucessful compiler run
[error] one error found
[error] (core/compile:doc) Scaladoc generation failed
[error] Total time: 67 s, completed Jan 6, 2014 2:20:51 PM
```

In theory a no-argument constructor ought not to differ from one with a single argument that has a default value, but in practice there seems to be an issue.
 2014-01-06 14:52:15 -08:00
Dan Crankshaw 86404dac74 Merge pull request #127 from jegonzal/MapByPartition 
Adding mapEdges and mapTriplets by Partition
 2014-01-04 14:55:54 -08:00
Joey e68cdb1b82 Merge pull request #124 from jianpingjwang/master 
refactor and bug fix
 2014-01-04 13:46:02 -08:00
Joseph E. Gonzalez 6592be2594 slightly more efficient map operation 2014-01-04 13:13:20 -08:00
Joseph E. Gonzalez 0b3efbcf62 Adding partition level mapEdges and mapTriplets. This is necessary to support computation with random number generation. 2014-01-04 13:13:20 -08:00
Joey 280ddf64bd Merge pull request #121 from ankurdave/more-simplify 
Simplify GraphImpl internals further
 2014-01-04 12:54:41 -08:00
Ankur Dave cfab8f2062 Revert "Create VertexPlacement on demand in VTableReplicated" 
This reverts commit 32f957f331.

Conflicts:
	graph/src/main/scala/org/apache/spark/graph/impl/GraphImpl.scala
	graph/src/main/scala/org/apache/spark/graph/impl/ReplicatedVertexView.scala
 2014-01-03 18:01:13 -07:00
Reynold Xin dc9cb83e86 Merge pull request #126 from jegonzal/FixingPersist 
Fixing Persist Behavior
 2014-01-01 13:28:34 -08:00
Joseph E. Gonzalez 2f2524fd11 Addressing issue in compute where compute is invoked instead of iterator on the parent RDD. 2013-12-31 21:37:51 -08:00
Joseph E. Gonzalez 3d93d73396 Fixing the persist behavior of the Vertex and Edge RDDs to persist the current RDD and not the parent. 2013-12-31 15:19:01 -08:00
Joey 32d6ae9d9c Merge pull request #120 from ankurdave/subgraph-reuses-view 
Reuse VTableReplicated in GraphImpl.subgraph
 2013-12-31 13:51:07 -08:00
Jianping J Wang 6e50df6255 Update SvdppSuite.scala 2013-12-31 22:02:16 +08:00
Jianping J Wang 61e6671f5a fix test bug 2013-12-31 22:01:02 +08:00
Jianping J Wang 12c26d7fb9 Update Svdpp.scala 2013-12-31 17:18:15 +08:00
Jianping J Wang ab7b8ce13e Update Svdpp.scala 2013-12-31 17:11:20 +08:00
Jianping J Wang 4a30f69b25 update svdpp test 2013-12-31 17:00:59 +08:00
Jianping J Wang 779c66ae4e refactor and fix bugs 2013-12-31 16:59:05 +08:00
Jianping J Wang 600421d8bc update 2013-12-30 23:42:55 +08:00
Jianping J Wang 29fe6bdaa2 refactor and bug fix 2013-12-30 23:41:15 +08:00
Reynold Xin 44e4205ac5 Merge pull request #116 from jianpingjwang/master 
remove unused variables and fix a bug
 2013-12-22 11:44:18 -08:00
Wang Jianping J e64a794a44 Update Svdpp.scala 2013-12-22 18:20:49 +08:00
Wang Jianping J b4bce88a2d Update Svdpp.scala 2013-12-21 16:08:19 +08:00
Wang Jianping J 49eb0f1351 Update Svdpp.scala 2013-12-21 15:51:05 +08:00
Wang Jianping J f986e4a136 Update Svdpp.scala 2013-12-21 13:18:11 +08:00
Wang Jianping J 47eefd30ed add javadoc 2013-12-21 13:02:59 +08:00
Ankur Dave 17311359c4 Rename VTableReplicated -> ReplicatedVertexView 2013-12-20 15:58:40 -08:00
Ankur Dave 32f957f331 Create VertexPlacement on demand in VTableReplicated 2013-12-20 15:37:33 -08:00
Ankur Dave 6d1bf0d78d Merge branch 'subgraph-reuses-view' into HEAD 2013-12-20 14:51:32 -08:00
Ankur Dave cd01539d6f Remove unused EdgePartition.filter 2013-12-20 13:38:19 -08:00
Ankur Dave 6bb077cd3d Reuse VTableReplicated in GraphImpl.subgraph 2013-12-20 13:38:19 -08:00
Reynold Xin 4797c227ff Merge pull request #118 from ankurdave/VertexPartitionSuite 
Test VertexPartition and fix bugs
 2013-12-20 13:30:39 -08:00
Ankur Dave 32508e20d4 Test VertexPartition and fix bugs 2013-12-20 13:00:06 -08:00
Reynold Xin ac70b8f234 Merge pull request #117 from ankurdave/more-tests 
More tests
 2013-12-20 10:56:10 -08:00
Ankur Dave efc765cf1a Test all methods on EdgePartition 2013-12-20 01:30:33 -08:00
Ankur Dave 752c0106e8 Test EdgePartition2D 2013-12-20 01:06:49 -08:00
Wang Jianping J 343d8977aa remove unused variable and fix a bug 2013-12-20 16:57:24 +08:00
Ankur Dave c3ec91a462 Write mapTriplets test 2013-12-20 00:30:21 -08:00
Reynold Xin 45310d4a8b Merge pull request #115 from ankurdave/test-reorg 
Reorganize unit tests; improve GraphSuite test coverage
 2013-12-19 22:08:20 -08:00
Ankur Dave 95381dfb94 Split AnalyticsSuite into algorithms suites 2013-12-19 20:46:03 -08:00
Ankur Dave a69465b1fa Split VertexRDD tests; fix #114 2013-12-19 20:32:30 -08:00
Ankur Dave da9f5e3fc0 Split GraphSuite; simplify LocalSparkContext 2013-12-19 19:49:07 -08:00
Ankur Dave da301b57fc Merge pull request #112 from amatsukawa/scc 
Strongly connected component algorithm
 2013-12-18 20:03:29 -08:00
Akihiro Matsukawa 989f22b365 added scc tests 2013-12-18 18:22:53 -08:00
Akihiro Matsukawa b1db1b6c84 scc algorithm 2013-12-18 18:22:43 -08:00
Ankur Dave a645ef633d Merge pull request #48 from amatsukawa/add_project_to_graph 
Add mask operation on graph and filter graph primitive
 2013-12-18 16:07:52 -08:00
Akihiro Matsukawa d7ebff0272 Merge pull request #1 from ankurdave/add_project_to_graph 
Merge current master and reimplement Graph.mask using innerJoin
 2013-12-18 15:38:48 -08:00
Ankur Dave 0f137e8b75 Reimplement Graph.mask using innerJoin 2013-12-18 13:01:13 -08:00
Ankur Dave 9193a8f788 Merge remote-tracking branch 'upstream/master' into add_project_to_graph 
Conflicts:
	graph/src/main/scala/org/apache/spark/graph/Graph.scala
	graph/src/main/scala/org/apache/spark/graph/impl/GraphImpl.scala
 2013-12-18 13:00:58 -08:00
Joey 3fd2e09ffb Merge pull request #104 from jianpingjwang/master 
SVD++ demo
 2013-12-18 12:52:36 -08:00
Wang Jianping J 06581b6a96 Update Svdpp.scala 2013-12-18 06:51:37 +08:00