Semih Salihoglu
1f4c7f7ecc
Graph primitives2
...
Hi guys,
I'm following Joey and Ankur's suggestions to add collectEdges and pickRandomVertex. I'm also adding the tests for collectEdges and refactoring one method getCycleGraph in GraphOpsSuite.scala.
Thank you,
semih
Author: Semih Salihoglu <semihsalihoglu@gmail.com>
Closes #580 from semihsalihoglu/GraphPrimitives2 and squashes the following commits:
937d3ec [Semih Salihoglu] - Fixed the scalastyle errors.
a69a152 [Semih Salihoglu] - Adding collectEdges and pickRandomVertices. - Adding tests for collectEdges. - Refactoring a getCycle utility function for GraphOpsSuite.scala.
41265a6 [Semih Salihoglu] - Adding collectEdges and pickRandomVertex. - Adding tests for collectEdges. - Recycling a getCycle utility test file.
2014-02-24 22:42:30 -08:00
Prashant Sharma
919bd7f669
Merge pull request #567 from ScrapCodes/style2.
...
SPARK-1058, Fix Style Errors and Add Scala Style to Spark Build. Pt 2
Continuation of PR #557
With this all scala style errors are fixed across the code base !!
The reason for creating a separate PR was to not interrupt an already reviewed and ready to merge PR. Hope this gets reviewed soon and merged too.
Author: Prashant Sharma <prashant.s@imaginea.com>
Closes #567 and squashes the following commits:
3b1ec30 [Prashant Sharma] scala style fixes
2014-02-09 22:17:52 -08:00
Patrick Wendell
b69f8b2a01
Merge pull request #557 from ScrapCodes/style. Closes #557 .
...
SPARK-1058, Fix Style Errors and Add Scala Style to Spark Build.
Author: Patrick Wendell <pwendell@gmail.com>
Author: Prashant Sharma <scrapcodes@gmail.com>
== Merge branch commits ==
commit 1a8bd1c059b842cb95cc246aaea74a79fec684f4
Author: Prashant Sharma <scrapcodes@gmail.com>
Date: Sun Feb 9 17:39:07 2014 +0530
scala style fixes
commit f91709887a8e0b608c5c2b282db19b8a44d53a43
Author: Patrick Wendell <pwendell@gmail.com>
Date: Fri Jan 24 11:22:53 2014 -0800
Adding scalastyle snapshot
2014-02-09 10:09:19 -08:00
Mark Hamstra
c2341c92bb
Merge pull request #542 from markhamstra/versionBump. Closes #542 .
...
Version number to 1.0.0-SNAPSHOT
Since 0.9.0-incubating is done and out the door, we shouldn't be building 0.9.0-incubating-SNAPSHOT anymore.
@pwendell
Author: Mark Hamstra <markhamstra@gmail.com>
== Merge branch commits ==
commit 1b00a8a7c1a7f251b4bb3774b84b9e64758eaa71
Author: Mark Hamstra <markhamstra@gmail.com>
Date: Wed Feb 5 09:30:32 2014 -0800
Version number to 1.0.0-SNAPSHOT
2014-02-08 16:00:43 -08:00
Jianping J Wang
a5a513e25e
Add jblas dependency
2014-01-23 19:48:39 +08:00
Jianping J Wang
cc0fd33177
Replace commons-math with jblas
2014-01-23 19:44:30 +08:00
Sean Owen
fd0c5b8c57
Depend on Commons Math explicitly instead of accidentally getting it from Hadoop (which stops working in 2.2.x) and also use the newer commons-math3
2014-01-22 22:25:49 +00:00
Patrick Wendell
5fecd2516d
Merge pull request #441 from pwendell/graphx-build
...
GraphX shouldn't list Spark as provided.
I noticed this when building an application against GraphX to audit the released artifacts.
2014-01-15 11:15:07 -08:00
Patrick Wendell
9259d706be
GraphX shouldn't list Spark as provided
2014-01-15 10:46:37 -08:00
Reynold Xin
3d9e66d92a
Merge pull request #436 from ankurdave/VertexId-case
...
Rename VertexID -> VertexId in GraphX
2014-01-14 23:17:05 -08:00
Reynold Xin
dfb152446d
Fixed SVDPlusPlusSuite in Maven build.
2014-01-14 22:18:43 -08:00
Ankur Dave
f4d9019aa8
VertexID -> VertexId
2014-01-14 22:17:18 -08:00
Patrick Wendell
23034798d7
Add missing header files
2014-01-14 01:17:13 -08:00
Joseph E. Gonzalez
80e73ed000
Adding minimal additional functionality to EdgeRDD
2014-01-13 22:56:57 -08:00
Ankur Dave
af645be5b8
Fix all code examples in guide
2014-01-13 22:29:45 -08:00
Ankur Dave
76ebdae798
Fix bug in GraphLoader.edgeListFile that caused srcId > dstId
2014-01-13 22:20:45 -08:00
Ankur Dave
c6dbfd1694
Edge object must be public for Edge case class
2014-01-13 22:08:44 -08:00
Ankur Dave
c28e5a08ee
Improve scaladoc links
2014-01-13 21:11:39 -08:00
Ankur Dave
c6023bee60
Fix infinite loop in GraphGenerators.generateRandomEdges
...
The loop occurred when numEdges < numVertices. This commit fixes it by
allowing generateRandomEdges to generate a multigraph.
2014-01-13 21:02:37 -08:00
Ankur Dave
84d6af8021
Make Graph{,Impl,Ops} serializable to work around capture
2014-01-13 21:02:37 -08:00
Ankur Dave
d4d9ece1af
Remove Graph.statistics and GraphImpl.printLineage
2014-01-13 21:02:37 -08:00
Reynold Xin
0b18bfba1a
Updated doc for PageRank.
2014-01-13 18:51:04 -08:00
Reynold Xin
9317286b72
More cleanup.
2014-01-13 18:45:35 -08:00
Reynold Xin
8e5c732430
Moved SVDPlusPlusConf into SVDPlusPlus object itself.
2014-01-13 18:45:20 -08:00
Reynold Xin
1dce9ce446
Moved PartitionStrategy's into an object.
2014-01-13 18:32:04 -08:00
Reynold Xin
ae06d2c22f
Updated GraphGenerator.
2014-01-13 18:31:49 -08:00
Reynold Xin
87f335db78
Made more things private.
2014-01-13 18:30:26 -08:00
Reynold Xin
a4e12af7aa
Merge branch 'graphx' of github.com:ankurdave/incubator-spark into graphx
...
Conflicts:
graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala
2014-01-13 17:42:59 -08:00
Reynold Xin
02a8f54bfa
Miscel doc update.
2014-01-13 17:40:36 -08:00
Joseph E. Gonzalez
cfe4a29dcb
Improvements in example code for the programming guide as well as adding serialization support for GraphImpl to address issues with failed closure capture.
2014-01-13 17:18:31 -08:00
Ankur Dave
ae4b75d94a
Add EdgeDirection.Either and use it to fix CC bug
...
The bug was due to a misunderstanding of the activeSetOpt parameter to
Graph.mapReduceTriplets. Passing EdgeDirection.Both causes
mapReduceTriplets to run only on edges with *both* vertices in the
active set. This commit adds EdgeDirection.Either, which causes
mapReduceTriplets to run on edges with *either* vertex in the active
set. This is what connected components needed.
2014-01-13 17:03:03 -08:00
Ankur Dave
1bd5cefcae
Remove aggregateNeighbors
2014-01-13 17:03:03 -08:00
Ankur Dave
8038da2328
Merge pull request #2 from jegonzal/GraphXCCIssue
...
Improving documentation and identifying potential bug in CC calculation.
2014-01-13 14:59:30 -08:00
Joseph E. Gonzalez
80e4d98dc6
Improving documentation and identifying potential bug in CC calculation.
2014-01-13 13:40:16 -08:00
Ankur Dave
9fe88627b5
Improve EdgeRDD scaladoc
2014-01-13 13:16:41 -08:00
Ankur Dave
ea69cff711
Further improve VertexRDD scaladocs
2014-01-13 12:52:52 -08:00
Ankur Dave
d691e9f47e
Move algorithms to GraphOps
2014-01-12 21:47:16 -08:00
Ankur Dave
20c509b805
Add TriangleCount example
2014-01-12 21:41:32 -08:00
Joseph E. Gonzalez
2216319f48
adding Pregel as an operator in GraphOps and cleaning up documentation of GraphOps
2014-01-12 21:26:37 -08:00
Ankur Dave
7a4bb863c7
Add connected components example to doc
2014-01-12 16:58:18 -08:00
Ankur Dave
5e35d39e0f
Add PageRank example and data
2014-01-12 13:10:53 -08:00
Ankur Dave
f096f4eaf1
Link methods in programming guide; document VertexID
2014-01-12 10:55:29 -08:00
Ankur Dave
02771aa087
Make EdgeDirection val instead of case object for Java compat.
2014-01-11 13:15:46 -08:00
Ankur Dave
574c0d28c2
Use SparkConf in GraphX tests (via LocalSparkContext)
2014-01-11 12:39:30 -08:00
Ankur Dave
55101f5821
One-line Scaladoc comments in Edge and EdgeDirection
2014-01-11 12:35:41 -08:00
Ankur Dave
64f73f73a0
Fix indent and use SparkConf in Analytics
2014-01-11 12:33:06 -08:00
Ankur Dave
732333d78e
Remove GraphLab
2014-01-11 11:49:35 -08:00
Ankur Dave
0b5c49ebad
Make nullValue and VertexSet package-private
2014-01-11 11:49:35 -08:00
Ankur Dave
feaa078022
algorithms -> lib
2014-01-11 00:30:10 -08:00
Ankur Dave
4f7ddf40fc
Optimize Edge.lexicographicOrdering
2014-01-11 00:15:01 -08:00
Ankur Dave
34496d6a9f
Move Analytics to algorithms and fix doc
2014-01-11 00:08:36 -08:00
Ankur Dave
2d7e8d8c48
Add GC note to GraphLab
2014-01-10 23:46:02 -08:00
Ankur Dave
c4fb6a87d3
Fix scaladoc warnings
2014-01-10 18:36:42 -08:00
Ankur Dave
41d6586e8e
Revert changes to Spark's (PrimitiveKey)OpenHashMap; copy PKOHM to graphx
2014-01-10 18:00:54 -08:00
Ankur Dave
85a6645d31
Add doc for Algorithms
2014-01-10 16:08:58 -08:00
Ankur Dave
04c20e7f4f
Minor cleanup to docs
2014-01-10 15:58:30 -08:00
Ankur Dave
1788729273
Move VertexIdToIndexMap into impl
2014-01-10 15:58:18 -08:00
Ankur Dave
57d7487d3d
Improve docs for VertexRDD
2014-01-10 15:48:20 -08:00
Ankur Dave
11dd35c28b
Clean up GraphGenerators
2014-01-10 15:23:32 -08:00
Ankur Dave
9e48af6dba
Remove unused HashUtils class
2014-01-10 15:22:57 -08:00
Ankur Dave
7bda997785
Improve docs for PartitionStrategy
2014-01-10 13:00:28 -08:00
Ankur Dave
eb4b46f8d1
Improve docs for GraphOps
2014-01-10 12:46:00 -08:00
Ankur Dave
9454fa1f6c
Remove duplicate method in GraphLoader and improve docs
2014-01-10 12:37:20 -08:00
Ankur Dave
37611e57f6
Improve docs for EdgeRDD, EdgeTriplet, and GraphLab
2014-01-10 12:37:03 -08:00
Ankur Dave
eee9bc0958
Remove commented-out perf files
2014-01-10 12:36:15 -08:00
Ankur Dave
c39ec3017f
Remove some commented code
2014-01-10 12:17:17 -08:00
Ankur Dave
5fcd2a61b4
Finish cleaning up Graph docs
2014-01-10 12:17:04 -08:00
Ankur Dave
4c114a7556
Start cleaning up Scaladocs in Graph and EdgeRDD
2014-01-10 11:37:54 -08:00
Ankur Dave
cfc10c74a3
Remove EdgeTriplet.{src,dst}Stale, which were unused
2014-01-10 10:43:23 -08:00
Ankur Dave
bf50e8c6cd
Remove commented code from Analytics
2014-01-10 10:37:04 -08:00
Ankur Dave
1b2aad918c
Update graphx/pom.xml to mirror mllib/pom.xml
2014-01-10 10:34:40 -08:00
Ankur Dave
729277ebc4
Undo 8b6b8ac87f
...
Getting unpersist right in GraphLab is tricky.
2014-01-10 01:53:28 -08:00
Ankur Dave
4cc550909a
graph -> graphx in log4j.properties
2014-01-10 00:59:59 -08:00
Ankur Dave
ba511f890e
Avoid recomputation by caching all multiply-used RDDs
2014-01-10 00:35:02 -08:00
Ankur Dave
8b6b8ac87f
Unpersist previous iterations in GraphLab
2014-01-10 00:34:08 -08:00
Ankur Dave
2578332f97
Add Graph.unpersistVertices()
2014-01-09 23:34:35 -08:00
Ankur Dave
8ae108f6c4
Unpersist previous iterations in Pregel
2014-01-09 23:25:35 -08:00
Ankur Dave
b7c92dded3
Add implicit algorithm methods for Graph; remove standalone PageRank
2014-01-09 20:44:28 -08:00
Ankur Dave
731f56f309
graph -> graphx
2014-01-09 14:31:33 -08:00