Matei Zaharia
34a47b8bc9
Update Scala version in docs
2013-04-07 20:27:03 -04:00
Matei Zaharia
b362df39ea
Merge pull request #552 from MLnick/master
...
Bumping version for Twitter Algebird to latest
2013-04-07 17:17:52 -07:00
Matei Zaharia
4b301902f5
Merge pull request #554 from andyk/scala2.9.3
...
Fixes SPARK-723 - Update build to Scala 2.9.3
2013-04-07 17:15:10 -07:00
Mridul Muralidharan
6798a09df8
Add support for building against hadoop2-yarn : adding new maven profile for it
2013-04-07 17:47:38 +05:30
Reynold Xin
583a5858e7
Added more logging to Analytics.
2013-04-07 15:45:28 +08:00
Reynold Xin
3728e1bc40
Code to run bagel vs graph experiments.
2013-04-07 15:05:46 +08:00
shane-huang
df47b40b76
Shuffle Performance fix: Use netty embeded OIO file server instead of ConnectionManager
...
Shuffle Performance Optimization: do not send 0-byte block requests to reduce network messages
change reference from io.Source to scala.io.Source to avoid looking into io.netty package
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
2013-04-07 14:37:12 +08:00
Holden Karau
ff2130a0ad
Retry failed ssh commands. This is especially useful during system startup when the hosts may not have yet come on-line but can be useful at other points for people with flakey connections
2013-04-06 00:35:50 -07:00
Reynold Xin
9eec317835
Minor cleanup.
2013-04-05 23:22:55 +08:00
Reynold Xin
822d9c5b70
Rename rawGraph to graph in Pregel.
2013-04-05 16:16:04 +08:00
Reynold Xin
d40c1d5122
Added unit test and fix a partitioner problem.
2013-04-05 15:33:50 +08:00
Joseph E. Gonzalez
14205548ce
Merged with master
...
Merge branch 'graph' of https://github.com/rxin/spark into graph
2013-04-04 23:51:06 -07:00
Joseph E. Gonzalez
092708e57e
better parsing of graph text files
2013-04-04 23:50:53 -07:00
Reynold Xin
e5dd61e720
Rename rawGraph to graph.
2013-04-05 14:47:39 +08:00
Reynold Xin
c973e564b9
Changed EdgeDirection from Enumeration to case classes.
2013-04-05 14:46:37 +08:00
Reynold Xin
34ef3e52dc
Merge branch 'graph' of github.com:rxin/spark into graph
2013-04-05 14:21:44 +08:00
Reynold Xin
9764e579b8
Minor cleanup.
2013-04-05 14:21:24 +08:00
Joseph E. Gonzalez
3d7a4b1fef
More realistic version of Pregel.
2013-04-04 23:17:27 -07:00
Reynold Xin
7134856351
numVertexPartitions and numEdgePartitions are now part of the
...
constructor and are immutable.
Also done some cleanups.
2013-04-05 12:54:59 +08:00
Joseph E. Gonzalez
d5b0f4dfa6
added a function to collect the neighborhood of a vertex as well as the skeleton of the program for classic pregel
2013-04-04 19:54:14 -07:00
Joseph E. Gonzalez
ad908f7545
added pregel pagerank
2013-04-04 19:31:14 -07:00
Joseph E. Gonzalez
d510045d8c
fixing the the silly bug in the dynamic pagerank gather function
2013-04-04 19:24:06 -07:00
Joseph E. Gonzalez
b537613570
added dynamic graphlab
2013-04-04 19:18:15 -07:00
Joseph E. Gonzalez
4a2b8aa557
added dynamic graphlab but I have not yet tested it.
2013-04-04 18:59:56 -07:00
Joseph E. Gonzalez
0667986c8e
Minor tweak to pregel semantics to give reverse edge illusion on send message.
2013-04-04 17:59:08 -07:00
Joseph E. Gonzalez
cb99fc193c
Added graphlab style implementation of Pregel and a more sophisticated version of mapReduceNeighborhoodFilter
2013-04-04 17:56:16 -07:00
Joseph E. Gonzalez
db45cf3a49
Fixing several bugs in mapReduceNeighborhood. First a map is used instead of a foreach which for mysterious reasons meant that the map never seems to be executed? Switching to a foreach causes a null pointer exception since the body of the foreach did not properly initialize the temporary EdgeWithVertex data structure.
2013-04-04 15:37:02 -07:00
Joseph E. Gonzalez
93eca18a62
fixing a silly bug whereby the pagerank equation was implemented incorrectly (divided by degree of dst instead of source).
2013-04-04 15:35:07 -07:00
Andy Konwinski
5555811bd5
Update build to Scala 2.9.3
2013-04-04 13:26:45 -07:00
Reynold Xin
1671abf47c
Implemented mapReduceNeighborhood in Graph and used that to implement
...
gather in GraphLab.
2013-04-05 01:47:59 +08:00
Joseph E. Gonzalez
28d0557fd8
added graph generator to run additional experiments
2013-04-03 23:23:07 -07:00
Joseph E. Gonzalez
91e1227edb
completed port of several analytics packages as well as analytics main
2013-04-03 23:04:29 -07:00
Joseph E. Gonzalez
39cac0ae65
Fixed iterateGAS to return a graph, added some minor features to graph and some additional todo items, and added the Analytics code from the internal SparkGraph prototype
2013-04-03 16:22:37 -07:00
Patrick Wendell
9b68ceaa26
SPARK-724 - Have Akka logging enabled by default for standalone daemons
...
See the JIRA for more details.
I was only able to test the bash version (don't have Windows)
so maybe check over that the syntax is correct there.
2013-04-03 14:29:46 -07:00
Joseph E. Gonzalez
ad73e5bdbb
Merging with downstream changes.
...
Merge branch 'graph' of https://github.com/rxin/spark into graph
2013-04-03 09:10:20 -07:00
Reynold Xin
4291da1481
Allow returning different vertex data type in updateVertices.
...
Please enter the commit message for your changes. Lines starting
2013-04-04 00:07:54 +08:00
Joseph E. Gonzalez
c649073b5f
merged with trunk
2013-04-03 08:47:49 -07:00
Joseph E. Gonzalez
0123c9d6a1
Changed GraphLab class to object and added graph loading from text file.
2013-04-03 08:43:08 -07:00
Reynold Xin
fe42ad41bb
Commit a working version.
2013-04-03 23:40:09 +08:00
Reynold Xin
cb0efe92d1
Oh wow it finally compiles!
2013-04-03 23:12:01 +08:00
Nick Pentreath
0f54344fd8
Bumping Algebird version in examples now that it supports JDK 1.6
2013-04-03 13:15:34 +02:00
Matei Zaharia
dfe98ca798
Merge pull request #550 from erikvanoosten/master
...
corrected Algebird example
2013-04-02 19:24:12 -07:00
Matei Zaharia
b5d78307ca
Merge pull request #551 from jey/python-bugfixes
...
Python bugfixes
2013-04-02 19:23:45 -07:00
Jey Kottalam
9a731f5a6d
Fix Python saveAsTextFile doctest to not expect order to be preserved
2013-04-02 11:59:20 -07:00
Jey Kottalam
20604001e2
Fix argv handling in Python transitive closure example
2013-04-02 11:59:07 -07:00
Reynold Xin
d63c895945
Partial checkin of graphlab module.
2013-04-03 00:42:33 +08:00
Erik van oosten
b5e60c3253
Corrected order of CountMinSketchMonoid arguments
2013-04-02 15:25:22 +03:00
Andrew xia
2b373dd07a
add properties default value null to fix sbt/sbt test errors
2013-04-02 12:11:14 +08:00
Reynold Xin
25c71b185d
Added a vertices method to Graph.
2013-04-02 01:26:20 +08:00
Reynold Xin
d7011b0f78
Added a Graph class that supports joining vertices with edges.
2013-04-02 01:17:44 +08:00
