Ankur Dave 15a564598f [SPARK-3427] [GraphX] Avoid active vertex tracking in static PageRank ... GraphX's current implementation of static (fixed iteration count) PageRank uses the Pregel API. This unnecessarily tracks active vertices, even though in static PageRank all vertices are always active. Active vertex tracking incurs the following costs: 1. A shuffle per iteration to ship the active sets to the edge partitions. 2. A hash table creation per iteration at each partition to index the active sets for lookup. 3. A hash lookup per edge to check whether the source vertex is active. I reimplemented static PageRank using the lower-level GraphX API instead of the Pregel API. In benchmarks on a 16-node m2.4xlarge cluster, this provided a 23% speedup (from 514 s to 397 s, mean over 3 trials) for 10 iterations of PageRank on a synthetic graph with 10M vertices and 1.27B edges. Author: Ankur Dave <ankurdave@gmail.com> Closes #2308 from ankurdave/SPARK-3427 and squashes the following commits: 449996a [Ankur Dave] Avoid unnecessary active vertex tracking in static PageRank		2014-09-12 14:08:38 -07:00
..
data	Fix all code examples in guide	2014-01-13 22:29:45 -08:00
src	[SPARK-3427] [GraphX] Avoid active vertex tracking in static PageRank	2014-09-12 14:08:38 -07:00
pom.xml	[SPARK-3397] Bump pom.xml version number of master branch to 1.2.0-SNAPSHOT	2014-09-06 15:04:50 -07:00