ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[Minor] Fix comments for GraphX 2D partitioning strategy

Browse source 
The sum of vertices on matrix (v0 to v11) is 12. And, I think one same block overlaps in this strategy.

This is minor PR, so I didn't file in JIRA.

Author: kj-ki <kikushima.kenji@lab.ntt.co.jp>

Closes #3904 from kj-ki/fix-partitionstrategy-comments and squashes the following commits:

79829d9 [kj-ki] Fix comments for 2D partitioning.
This commit is contained in:
kj-ki 2015-01-06 09:49:37 -08:00 committed by Ankur Dave
parent a6394bc2c0
commit 5e3ec11104
1 changed files with 3 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
graphx/src/main/scala/org/apache/spark/graphx/PartitionStrategy.scala
View file

@ -32,9 +32,9 @@ trait PartitionStrategy extends Serializable {
object PartitionStrategy { object PartitionStrategy {
/** /**
* Assigns edges to partitions using a 2D partitioning of the sparse edge adjacency matrix, * Assigns edges to partitions using a 2D partitioning of the sparse edge adjacency matrix,
* guaranteeing a `2 * sqrt(numParts)` bound on vertex replication. * guaranteeing a `2 * sqrt(numParts) - 1` bound on vertex replication.
* *
* Suppose we have a graph with 11 vertices that we want to partition * Suppose we have a graph with 12 vertices that we want to partition
* over 9 machines. We can use the following sparse matrix representation: * over 9 machines. We can use the following sparse matrix representation:
* *
* <pre> * <pre>
@ -61,7 +61,7 @@ object PartitionStrategy {
* that edges adjacent to `v11` can only be in the first column of blocks `(P0, P3, * that edges adjacent to `v11` can only be in the first column of blocks `(P0, P3,
* P6)` or the last * P6)` or the last
* row of blocks `(P6, P7, P8)`. As a consequence we can guarantee that `v11` will need to be * row of blocks `(P6, P7, P8)`. As a consequence we can guarantee that `v11` will need to be
* replicated to at most `2 * sqrt(numParts)` machines. * replicated to at most `2 * sqrt(numParts) - 1` machines.
* *
* Notice that `P0` has many edges and as a consequence this partitioning would lead to poor work * Notice that `P0` has many edges and as a consequence this partitioning would lead to poor work
* balance. To improve balance we first multiply each vertex id by a large prime to shuffle the * balance. To improve balance we first multiply each vertex id by a large prime to shuffle the