ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2670 commits 6 branches 181 tags 397 MiB
Commit graph

2670 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia adba773fab Fix passing of superstep in Bagel to avoid seeing new values of the 
superstep value upon recomputation, and set the default storage level in
Bagel to MEMORY_AND_DISK
 2013-04-08 17:34:38 -04:00
Matei Zaharia eed54a25d8 Merge pull request #553 from pwendell/akka-standalone 
SPARK-724 - Have Akka logging enabled by default for standalone daemons
 2013-04-08 09:44:30 -07:00
Matei Zaharia 65caa8f711 Merge remote-tracking branch 'jey/bump-development-version-to-0.8.0' 
Conflicts:
	docs/_config.yml
	project/SparkBuild.scala
 2013-04-08 12:43:17 -04:00
Matei Zaharia 054feb6448 Fixed a bug with zip 2013-04-07 21:15:21 -04:00
Matei Zaharia b5900d47b1 Fix compile warning 2013-04-07 20:55:42 -04:00
Matei Zaharia 1cb3eb9762 Merge remote-tracking branch 'kalpit/master' 
Conflicts:
	project/SparkBuild.scala
 2013-04-07 20:54:18 -04:00
Patrick Wendell b496decf0a Updating based on code review 2013-04-07 17:44:48 -07:00
Matei Zaharia a1586412d6 Updated link to SBT 2013-04-07 20:31:19 -04:00
Matei Zaharia 6962d40b44 Fix deprecated warning 2013-04-07 20:27:33 -04:00
Matei Zaharia 34a47b8bc9 Update Scala version in docs 2013-04-07 20:27:03 -04:00
Matei Zaharia b362df39ea Merge pull request #552 from MLnick/master 
Bumping version for Twitter Algebird to latest
 2013-04-07 17:17:52 -07:00
Matei Zaharia 4b301902f5 Merge pull request #554 from andyk/scala2.9.3 
Fixes SPARK-723 - Update build to Scala 2.9.3
 2013-04-07 17:15:10 -07:00
Mridul Muralidharan 6798a09df8 Add support for building against hadoop2-yarn : adding new maven profile for it 2013-04-07 17:47:38 +05:30
Holden Karau ff2130a0ad Retry failed ssh commands. This is especially useful during system startup when the hosts may not have yet come on-line but can be useful at other points for people with flakey connections 2013-04-06 00:35:50 -07:00
Andy Konwinski 5555811bd5 Update build to Scala 2.9.3 2013-04-04 13:26:45 -07:00
Patrick Wendell 9b68ceaa26 SPARK-724 - Have Akka logging enabled by default for standalone daemons 
See the JIRA for more details.

I was only able to test the bash version (don't have Windows)
so maybe check over that the syntax is correct there.
 2013-04-03 14:29:46 -07:00
Nick Pentreath 0f54344fd8 Bumping Algebird version in examples now that it supports JDK 1.6 2013-04-03 13:15:34 +02:00
Matei Zaharia dfe98ca798 Merge pull request #550 from erikvanoosten/master 
corrected Algebird example
 2013-04-02 19:24:12 -07:00
Matei Zaharia b5d78307ca Merge pull request #551 from jey/python-bugfixes 
Python bugfixes
 2013-04-02 19:23:45 -07:00
Jey Kottalam 9a731f5a6d Fix Python saveAsTextFile doctest to not expect order to be preserved 2013-04-02 11:59:20 -07:00
Jey Kottalam 20604001e2 Fix argv handling in Python transitive closure example 2013-04-02 11:59:07 -07:00
Erik van oosten b5e60c3253 Corrected order of CountMinSketchMonoid arguments 2013-04-02 15:25:22 +03:00
Matei Zaharia 2be2295ff2 Merge pull request #548 from markhamstra/getWritableClass_filter 
Fixed broken filter in getWritableClass[T]
 2013-03-31 18:09:14 -07:00
Mark Hamstra e215f67923 Correct sense of 'filter out' in comment. 2013-03-31 08:00:13 -07:00
Mark Hamstra 8bcdc64005 Fixed broken filter in getWritableClass[T] 2013-03-30 22:09:52 -07:00
Matei Zaharia 9831bc1a09 Merge pull request #539 from cgrothaus/fix-webui-workdirpath 
Bugfix: WorkerWebUI must respect workDirPath from Worker
 2013-03-29 22:16:22 -07:00
Matei Zaharia 3cc8ab6e29 Merge pull request #541 from stephenh/shufflecoalesce 
Add a shuffle parameter to coalesce.
 2013-03-29 22:14:07 -07:00
Matei Zaharia cad507adaf Merge pull request #547 from jey/maven-streaming-tests-initialization-fix 
Move streaming test initialization into 'before' blocks
 2013-03-29 22:13:12 -07:00
Matei Zaharia a98996d1fe Merge pull request #545 from ash211/patch-1 
Don't use deprecated Application in example
 2013-03-29 22:12:15 -07:00
Matei Zaharia 104c69414f Merge pull request #546 from ash211/patch-2 
Update tuning.md
 2013-03-29 22:11:50 -07:00
Jey Kottalam bc8ba222ff Bump development version to 0.8.0 2013-03-28 15:42:01 -07:00
Andrew Ash e8f3669c63 Update tuning.md 
Make the example more compilable
 2013-03-28 19:17:39 -03:00
Jey Kottalam b569b3f200 Move streaming test initialization into 'before' blocks 2013-03-28 15:08:41 -07:00
Andrew Ash 4e2c965383 Don't use deprecated Application in example 
As of 2.9.0 extending from Application is not recommended

http://www.scala-lang.org/api/2.9.3/index.html#scala.Application
 2013-03-28 17:47:37 -03:00
kalpit aa9134f72a spark instance number must be present in log filename to prevent multiple workers from overriding each other's logs 2013-03-26 17:49:30 -07:00
kalpit f08db010d3 added SPARK_WORKER_INSTANCES : allows spawning multiple worker instances/processes on every slave machine 2013-03-26 17:49:30 -07:00
kalpit f0164e5047 upgraded sbt version, sbt plugins and some library dependencies to latest stable version 2013-03-26 17:49:29 -07:00
Matei Zaharia bc36ee4fbb Merge pull request #543 from holdenk/master 
Re-enable deprecation warnings and fix deprecated warning.
 2013-03-26 15:05:13 -07:00
Andy Konwinski 6c18131487 Add comment to README that 2.10 not yet supported 2013-03-26 14:28:08 -07:00
Holden Karau 1f5381119f method first in trait IterableLike is deprecated: use `head' instead 2013-03-24 19:19:40 -07:00
Holden Karau 8456d673e2 Re-enable deprecation warnings since there are only two 2013-03-24 17:30:23 -07:00
Holden Karau e104a76016 Makes the syntax highlighting on the build file not broken in emacs. 2013-03-24 16:16:05 -07:00
Holden Karau f5df729b12 Explicitly catch all throwables (warning in 2.10) 2013-03-24 16:15:32 -07:00
Stephen Haberman dd854d5b9f Use Boolean in the Java API, and != for assert. 2013-03-23 11:49:45 -05:00
Stephen Haberman 4ca273edc4 Merge branch 'master' into shufflecoalesce 
Conflicts:
	core/src/test/scala/spark/RDDSuite.scala
 2013-03-23 11:45:45 -05:00
Matei Zaharia b8949cab88 Merge pull request #505 from stephenh/volatile 
Make Executor fields volatile since they're read from the thread pool.
 2013-03-23 07:19:34 -07:00
Matei Zaharia fd53f2fc7b Merge pull request #510 from markhamstra/WithThing 
mapWith, flatMapWith and filterWith
 2013-03-23 07:13:21 -07:00
Stephen Haberman 00170eb0b9 Fix are/our typo. 2013-03-22 12:59:08 -05:00
Stephen Haberman 1c67c7dfd1 Add a shuffle parameter to coalesce. 
This is useful for when you want just 1 output file (part-00000) but
still up the upstream RDD to be computed in parallel.
 2013-03-22 08:54:44 -05:00
Christoph Grothaus 445f387ef4 Bugfix: WorkerWebUI must respect workDirPath from Worker 2013-03-22 11:08:40 +01:00