Matei Zaharia
adba773fab
Fix passing of superstep in Bagel to avoid seeing new values of the
...
superstep value upon recomputation, and set the default storage level in
Bagel to MEMORY_AND_DISK
2013-04-08 17:34:38 -04:00
Matei Zaharia
eed54a25d8
Merge pull request #553 from pwendell/akka-standalone
...
SPARK-724 - Have Akka logging enabled by default for standalone daemons
2013-04-08 09:44:30 -07:00
Matei Zaharia
65caa8f711
Merge remote-tracking branch 'jey/bump-development-version-to-0.8.0'
...
Conflicts:
docs/_config.yml
project/SparkBuild.scala
2013-04-08 12:43:17 -04:00
Matei Zaharia
054feb6448
Fixed a bug with zip
2013-04-07 21:15:21 -04:00
Matei Zaharia
b5900d47b1
Fix compile warning
2013-04-07 20:55:42 -04:00
Matei Zaharia
1cb3eb9762
Merge remote-tracking branch 'kalpit/master'
...
Conflicts:
project/SparkBuild.scala
2013-04-07 20:54:18 -04:00
Patrick Wendell
b496decf0a
Updating based on code review
2013-04-07 17:44:48 -07:00
Matei Zaharia
a1586412d6
Updated link to SBT
2013-04-07 20:31:19 -04:00
Matei Zaharia
6962d40b44
Fix deprecated warning
2013-04-07 20:27:33 -04:00
Matei Zaharia
34a47b8bc9
Update Scala version in docs
2013-04-07 20:27:03 -04:00
Matei Zaharia
b362df39ea
Merge pull request #552 from MLnick/master
...
Bumping version for Twitter Algebird to latest
2013-04-07 17:17:52 -07:00
Matei Zaharia
4b301902f5
Merge pull request #554 from andyk/scala2.9.3
...
Fixes SPARK-723 - Update build to Scala 2.9.3
2013-04-07 17:15:10 -07:00
Mridul Muralidharan
6798a09df8
Add support for building against hadoop2-yarn : adding new maven profile for it
2013-04-07 17:47:38 +05:30
Reynold Xin
583a5858e7
Added more logging to Analytics.
2013-04-07 15:45:28 +08:00
Reynold Xin
3728e1bc40
Code to run bagel vs graph experiments.
2013-04-07 15:05:46 +08:00
shane-huang
df47b40b76
Shuffle Performance fix: Use netty embeded OIO file server instead of ConnectionManager
...
Shuffle Performance Optimization: do not send 0-byte block requests to reduce network messages
change reference from io.Source to scala.io.Source to avoid looking into io.netty package
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
2013-04-07 14:37:12 +08:00
Holden Karau
ff2130a0ad
Retry failed ssh commands. This is especially useful during system startup when the hosts may not have yet come on-line but can be useful at other points for people with flakey connections
2013-04-06 00:35:50 -07:00
Reynold Xin
9eec317835
Minor cleanup.
2013-04-05 23:22:55 +08:00
Reynold Xin
822d9c5b70
Rename rawGraph to graph in Pregel.
2013-04-05 16:16:04 +08:00
Reynold Xin
d40c1d5122
Added unit test and fix a partitioner problem.
2013-04-05 15:33:50 +08:00
Joseph E. Gonzalez
14205548ce
Merged with master
...
Merge branch 'graph' of https://github.com/rxin/spark into graph
2013-04-04 23:51:06 -07:00
Joseph E. Gonzalez
092708e57e
better parsing of graph text files
2013-04-04 23:50:53 -07:00
Reynold Xin
e5dd61e720
Rename rawGraph to graph.
2013-04-05 14:47:39 +08:00
Reynold Xin
c973e564b9
Changed EdgeDirection from Enumeration to case classes.
2013-04-05 14:46:37 +08:00
Reynold Xin
34ef3e52dc
Merge branch 'graph' of github.com:rxin/spark into graph
2013-04-05 14:21:44 +08:00
Reynold Xin
9764e579b8
Minor cleanup.
2013-04-05 14:21:24 +08:00
Joseph E. Gonzalez
3d7a4b1fef
More realistic version of Pregel.
2013-04-04 23:17:27 -07:00
Reynold Xin
7134856351
numVertexPartitions and numEdgePartitions are now part of the
...
constructor and are immutable.
Also done some cleanups.
2013-04-05 12:54:59 +08:00
Joseph E. Gonzalez
d5b0f4dfa6
added a function to collect the neighborhood of a vertex as well as the skeleton of the program for classic pregel
2013-04-04 19:54:14 -07:00
Joseph E. Gonzalez
ad908f7545
added pregel pagerank
2013-04-04 19:31:14 -07:00
Joseph E. Gonzalez
d510045d8c
fixing the the silly bug in the dynamic pagerank gather function
2013-04-04 19:24:06 -07:00
Joseph E. Gonzalez
b537613570
added dynamic graphlab
2013-04-04 19:18:15 -07:00
Joseph E. Gonzalez
4a2b8aa557
added dynamic graphlab but I have not yet tested it.
2013-04-04 18:59:56 -07:00
Joseph E. Gonzalez
0667986c8e
Minor tweak to pregel semantics to give reverse edge illusion on send message.
2013-04-04 17:59:08 -07:00
Joseph E. Gonzalez
cb99fc193c
Added graphlab style implementation of Pregel and a more sophisticated version of mapReduceNeighborhoodFilter
2013-04-04 17:56:16 -07:00
Joseph E. Gonzalez
db45cf3a49
Fixing several bugs in mapReduceNeighborhood. First a map is used instead of a foreach which for mysterious reasons meant that the map never seems to be executed? Switching to a foreach causes a null pointer exception since the body of the foreach did not properly initialize the temporary EdgeWithVertex data structure.
2013-04-04 15:37:02 -07:00
Joseph E. Gonzalez
93eca18a62
fixing a silly bug whereby the pagerank equation was implemented incorrectly (divided by degree of dst instead of source).
2013-04-04 15:35:07 -07:00
Andy Konwinski
5555811bd5
Update build to Scala 2.9.3
2013-04-04 13:26:45 -07:00
Reynold Xin
1671abf47c
Implemented mapReduceNeighborhood in Graph and used that to implement
...
gather in GraphLab.
2013-04-05 01:47:59 +08:00
Joseph E. Gonzalez
28d0557fd8
added graph generator to run additional experiments
2013-04-03 23:23:07 -07:00
Joseph E. Gonzalez
91e1227edb
completed port of several analytics packages as well as analytics main
2013-04-03 23:04:29 -07:00
Joseph E. Gonzalez
39cac0ae65
Fixed iterateGAS to return a graph, added some minor features to graph and some additional todo items, and added the Analytics code from the internal SparkGraph prototype
2013-04-03 16:22:37 -07:00
Patrick Wendell
9b68ceaa26
SPARK-724 - Have Akka logging enabled by default for standalone daemons
...
See the JIRA for more details.
I was only able to test the bash version (don't have Windows)
so maybe check over that the syntax is correct there.
2013-04-03 14:29:46 -07:00
Joseph E. Gonzalez
ad73e5bdbb
Merging with downstream changes.
...
Merge branch 'graph' of https://github.com/rxin/spark into graph
2013-04-03 09:10:20 -07:00
Reynold Xin
4291da1481
Allow returning different vertex data type in updateVertices.
...
Please enter the commit message for your changes. Lines starting
2013-04-04 00:07:54 +08:00
Joseph E. Gonzalez
c649073b5f
merged with trunk
2013-04-03 08:47:49 -07:00
Joseph E. Gonzalez
0123c9d6a1
Changed GraphLab class to object and added graph loading from text file.
2013-04-03 08:43:08 -07:00
Reynold Xin
fe42ad41bb
Commit a working version.
2013-04-03 23:40:09 +08:00
Reynold Xin
cb0efe92d1
Oh wow it finally compiles!
2013-04-03 23:12:01 +08:00
Nick Pentreath
0f54344fd8
Bumping Algebird version in examples now that it supports JDK 1.6
2013-04-03 13:15:34 +02:00
