ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
28661 commits 6 branches 181 tags 397 MiB
Commit graph

326 commits

Author SHA1 Message Date
Mark Hamstra c2341c92bb Merge pull request #542 from markhamstra/versionBump. Closes #542. 
Version number to 1.0.0-SNAPSHOT

Since 0.9.0-incubating is done and out the door, we shouldn't be building 0.9.0-incubating-SNAPSHOT anymore.

@pwendell

Author: Mark Hamstra <markhamstra@gmail.com>

== Merge branch commits ==

commit 1b00a8a7c1a7f251b4bb3774b84b9e64758eaa71
Author: Mark Hamstra <markhamstra@gmail.com>
Date:   Wed Feb 5 09:30:32 2014 -0800

    Version number to 1.0.0-SNAPSHOT
 2014-02-08 16:00:43 -08:00
Jianping J Wang a5a513e25e Add jblas dependency 2014-01-23 19:48:39 +08:00
Jianping J Wang cc0fd33177 Replace commons-math with jblas 2014-01-23 19:44:30 +08:00
Sean Owen fd0c5b8c57 Depend on Commons Math explicitly instead of accidentally getting it from Hadoop (which stops working in 2.2.x) and also use the newer commons-math3 2014-01-22 22:25:49 +00:00
Patrick Wendell 5fecd2516d Merge pull request #441 from pwendell/graphx-build 
GraphX shouldn't list Spark as provided.

I noticed this when building an application against GraphX to audit the released artifacts.
 2014-01-15 11:15:07 -08:00
Patrick Wendell 9259d706be GraphX shouldn't list Spark as provided 2014-01-15 10:46:37 -08:00
Reynold Xin 3d9e66d92a Merge pull request #436 from ankurdave/VertexId-case 
Rename VertexID -> VertexId in GraphX
 2014-01-14 23:17:05 -08:00
Reynold Xin dfb152446d Fixed SVDPlusPlusSuite in Maven build. 2014-01-14 22:18:43 -08:00
Ankur Dave f4d9019aa8 VertexID -> VertexId 2014-01-14 22:17:18 -08:00
Patrick Wendell 23034798d7 Add missing header files 2014-01-14 01:17:13 -08:00
Joseph E. Gonzalez 80e73ed000 Adding minimal additional functionality to EdgeRDD 2014-01-13 22:56:57 -08:00
Ankur Dave af645be5b8 Fix all code examples in guide 2014-01-13 22:29:45 -08:00
Ankur Dave 76ebdae798 Fix bug in GraphLoader.edgeListFile that caused srcId > dstId 2014-01-13 22:20:45 -08:00
Ankur Dave c6dbfd1694 Edge object must be public for Edge case class 2014-01-13 22:08:44 -08:00
Ankur Dave c28e5a08ee Improve scaladoc links 2014-01-13 21:11:39 -08:00
Ankur Dave c6023bee60 Fix infinite loop in GraphGenerators.generateRandomEdges 
The loop occurred when numEdges < numVertices. This commit fixes it by
allowing generateRandomEdges to generate a multigraph.
 2014-01-13 21:02:37 -08:00
Ankur Dave 84d6af8021 Make Graph{,Impl,Ops} serializable to work around capture 2014-01-13 21:02:37 -08:00
Ankur Dave d4d9ece1af Remove Graph.statistics and GraphImpl.printLineage 2014-01-13 21:02:37 -08:00
Reynold Xin 0b18bfba1a Updated doc for PageRank. 2014-01-13 18:51:04 -08:00
Reynold Xin 9317286b72 More cleanup. 2014-01-13 18:45:35 -08:00
Reynold Xin 8e5c732430 Moved SVDPlusPlusConf into SVDPlusPlus object itself. 2014-01-13 18:45:20 -08:00
Reynold Xin 1dce9ce446 Moved PartitionStrategy's into an object. 2014-01-13 18:32:04 -08:00
Reynold Xin ae06d2c22f Updated GraphGenerator. 2014-01-13 18:31:49 -08:00
Reynold Xin 87f335db78 Made more things private. 2014-01-13 18:30:26 -08:00
Reynold Xin a4e12af7aa Merge branch 'graphx' of github.com:ankurdave/incubator-spark into graphx 
Conflicts:
	graphx/src/main/scala/org/apache/spark/graphx/Pregel.scala
 2014-01-13 17:42:59 -08:00
Reynold Xin 02a8f54bfa Miscel doc update. 2014-01-13 17:40:36 -08:00
Joseph E. Gonzalez cfe4a29dcb Improvements in example code for the programming guide as well as adding serialization support for GraphImpl to address issues with failed closure capture. 2014-01-13 17:18:31 -08:00
Ankur Dave ae4b75d94a Add EdgeDirection.Either and use it to fix CC bug 
The bug was due to a misunderstanding of the activeSetOpt parameter to
Graph.mapReduceTriplets. Passing EdgeDirection.Both causes
mapReduceTriplets to run only on edges with *both* vertices in the
active set. This commit adds EdgeDirection.Either, which causes
mapReduceTriplets to run on edges with *either* vertex in the active
set. This is what connected components needed.
 2014-01-13 17:03:03 -08:00
Ankur Dave 1bd5cefcae Remove aggregateNeighbors 2014-01-13 17:03:03 -08:00
Ankur Dave 8038da2328 Merge pull request #2 from jegonzal/GraphXCCIssue 
Improving documentation and identifying potential bug in CC calculation.
 2014-01-13 14:59:30 -08:00
Joseph E. Gonzalez 80e4d98dc6 Improving documentation and identifying potential bug in CC calculation. 2014-01-13 13:40:16 -08:00
Ankur Dave 9fe88627b5 Improve EdgeRDD scaladoc 2014-01-13 13:16:41 -08:00
Ankur Dave ea69cff711 Further improve VertexRDD scaladocs 2014-01-13 12:52:52 -08:00
Ankur Dave d691e9f47e Move algorithms to GraphOps 2014-01-12 21:47:16 -08:00
Ankur Dave 20c509b805 Add TriangleCount example 2014-01-12 21:41:32 -08:00
Joseph E. Gonzalez 2216319f48 adding Pregel as an operator in GraphOps and cleaning up documentation of GraphOps 2014-01-12 21:26:37 -08:00
Ankur Dave 7a4bb863c7 Add connected components example to doc 2014-01-12 16:58:18 -08:00
Ankur Dave 5e35d39e0f Add PageRank example and data 2014-01-12 13:10:53 -08:00
Ankur Dave f096f4eaf1 Link methods in programming guide; document VertexID 2014-01-12 10:55:29 -08:00
Ankur Dave 02771aa087 Make EdgeDirection val instead of case object for Java compat. 2014-01-11 13:15:46 -08:00
Ankur Dave 574c0d28c2 Use SparkConf in GraphX tests (via LocalSparkContext) 2014-01-11 12:39:30 -08:00
Ankur Dave 55101f5821 One-line Scaladoc comments in Edge and EdgeDirection 2014-01-11 12:35:41 -08:00
Ankur Dave 64f73f73a0 Fix indent and use SparkConf in Analytics 2014-01-11 12:33:06 -08:00
Ankur Dave 732333d78e Remove GraphLab 2014-01-11 11:49:35 -08:00
Ankur Dave 0b5c49ebad Make nullValue and VertexSet package-private 2014-01-11 11:49:35 -08:00
Ankur Dave feaa078022 algorithms -> lib 2014-01-11 00:30:10 -08:00
Ankur Dave 4f7ddf40fc Optimize Edge.lexicographicOrdering 2014-01-11 00:15:01 -08:00
Ankur Dave 34496d6a9f Move Analytics to algorithms and fix doc 2014-01-11 00:08:36 -08:00
Ankur Dave 2d7e8d8c48 Add GC note to GraphLab 2014-01-10 23:46:02 -08:00
Ankur Dave c4fb6a87d3 Fix scaladoc warnings 2014-01-10 18:36:42 -08:00
Ankur Dave 41d6586e8e Revert changes to Spark's (PrimitiveKey)OpenHashMap; copy PKOHM to graphx 2014-01-10 18:00:54 -08:00
Ankur Dave 85a6645d31 Add doc for Algorithms 2014-01-10 16:08:58 -08:00
Ankur Dave 04c20e7f4f Minor cleanup to docs 2014-01-10 15:58:30 -08:00
Ankur Dave 1788729273 Move VertexIdToIndexMap into impl 2014-01-10 15:58:18 -08:00
Ankur Dave 57d7487d3d Improve docs for VertexRDD 2014-01-10 15:48:20 -08:00
Ankur Dave 11dd35c28b Clean up GraphGenerators 2014-01-10 15:23:32 -08:00
Ankur Dave 9e48af6dba Remove unused HashUtils class 2014-01-10 15:22:57 -08:00
Ankur Dave 7bda997785 Improve docs for PartitionStrategy 2014-01-10 13:00:28 -08:00
Ankur Dave eb4b46f8d1 Improve docs for GraphOps 2014-01-10 12:46:00 -08:00
Ankur Dave 9454fa1f6c Remove duplicate method in GraphLoader and improve docs 2014-01-10 12:37:20 -08:00
Ankur Dave 37611e57f6 Improve docs for EdgeRDD, EdgeTriplet, and GraphLab 2014-01-10 12:37:03 -08:00
Ankur Dave eee9bc0958 Remove commented-out perf files 2014-01-10 12:36:15 -08:00
Ankur Dave c39ec3017f Remove some commented code 2014-01-10 12:17:17 -08:00
Ankur Dave 5fcd2a61b4 Finish cleaning up Graph docs 2014-01-10 12:17:04 -08:00
Ankur Dave 4c114a7556 Start cleaning up Scaladocs in Graph and EdgeRDD 2014-01-10 11:37:54 -08:00
Ankur Dave cfc10c74a3 Remove EdgeTriplet.{src,dst}Stale, which were unused 2014-01-10 10:43:23 -08:00
Ankur Dave bf50e8c6cd Remove commented code from Analytics 2014-01-10 10:37:04 -08:00
Ankur Dave 1b2aad918c Update graphx/pom.xml to mirror mllib/pom.xml 2014-01-10 10:34:40 -08:00
Ankur Dave 729277ebc4 Undo 8b6b8ac87f 
Getting unpersist right in GraphLab is tricky.
 2014-01-10 01:53:28 -08:00
Ankur Dave 4cc550909a graph -> graphx in log4j.properties 2014-01-10 00:59:59 -08:00
Ankur Dave ba511f890e Avoid recomputation by caching all multiply-used RDDs 2014-01-10 00:35:02 -08:00
Ankur Dave 8b6b8ac87f Unpersist previous iterations in GraphLab 2014-01-10 00:34:08 -08:00
Ankur Dave 2578332f97 Add Graph.unpersistVertices() 2014-01-09 23:34:35 -08:00
Ankur Dave 8ae108f6c4 Unpersist previous iterations in Pregel 2014-01-09 23:25:35 -08:00
Ankur Dave b7c92dded3 Add implicit algorithm methods for Graph; remove standalone PageRank 2014-01-09 20:44:28 -08:00
Ankur Dave 731f56f309 graph -> graphx 2014-01-09 14:31:33 -08:00