ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2608 commits 6 branches 181 tags 397 MiB
Commit graph

2608 commits

This branch
This branch
All branches
Author SHA1 Message Date
Prashant Sharma 185bb9525a Manually merged scala-2.10 and master 2013-04-22 14:14:03 +05:30
Matei Zaharia 4b57f83209 Merge pull request #535 from ScrapCodes/scala-2.10-repl-port 
porting of repl to scala-2.10
 2013-04-20 10:40:07 -07:00
Prashant Sharma bf5fc07379 Added more tests 2013-04-19 13:51:16 +05:30
Matei Zaharia 17e076de80 Turn on forking in test JVMs to reduce the pressure on perm gen and code 
cache sizes due to having 2 instances of the Scala compiler and a bunch
of classloaders.
 2013-04-18 22:25:57 -07:00
Matei Zaharia 5d8a71c484 Merge pull request #570 from jey/increase-codecache-size 
Increase ReservedCodeCacheSize for sbt
 2013-04-16 19:48:02 -07:00
Jey Kottalam 6bfe4bf3eb Increase ReservedCodeCacheSize for sbt 2013-04-16 09:50:59 -07:00
Prashant Sharma 36ccb35371 Changed spark context as lazy val to become just val. 2013-04-16 11:13:51 +05:30
Matei Zaharia a64c107449 Make ShuffledRDD.prev transient 2013-04-15 16:41:51 -04:00
Prashant Sharma 19b0256ae4 Added standalone cluster repl suite 2013-04-15 19:49:40 +05:30
Matei Zaharia ec5e553b41 Merge pull request #558 from ash211/patch-jackson-conflict 
Don't pull in old versions of Jackson via hadoop-core
 2013-04-14 08:20:13 -07:00
Matei Zaharia c1c219e263 Merge pull request #564 from maspotts/master 
Allow latest scala in PATH, with SCALA_HOME as override (instead of vice-versa)
 2013-04-14 08:11:23 -07:00
Matei Zaharia c35d530bcf Fix compile error 2013-04-13 12:43:12 -04:00
Matei Zaharia 7c10b3e3cd Merge pull request #565 from andyk/master 
Update wording of section on RDD operations in quick start guide in docs
 2013-04-12 20:55:22 -07:00
Andy Konwinski 60a91b3b59 Update quick-start.md heading on Operations (not just Transformations). 2013-04-12 12:34:51 -07:00
Mike 6f68860891 Reversed the order of tests to find a scala executable (in the case when SPARK_LAUNCH_WITH_SCALA is defined): instead of checking in the PATH first, and only then (if not found) for SCALA_HOME, now we check for SCALA_HOME first, and only then (if not defined) do we look in the PATH. The advantage is that now if the user has a more recent (non-compatible) version of scala in her PATH, she can use SCALA_HOME to point to the older (compatible) version for use with spark. 
Suggested by Josh Rosen in this thread:

  https://groups.google.com/forum/?fromgroups=#!topic/spark-users/NC9JKvP8808
 2013-04-11 20:52:06 -07:00
Matei Zaharia 077ae0a197 Merge pull request #561 from ash211/patch-4 
Add details when BlockManager heartbeats time out
 2013-04-11 19:34:14 -07:00
Matei Zaharia e2aa87558d Merge branch 'master' of github.com:mesos/spark 2013-04-11 22:30:05 -04:00
Matei Zaharia ed336e0d44 Fix tests from different projects running in parallel in SBT 0.12 2013-04-11 22:29:37 -04:00
Andrew Ash 29d3440efb Add details when BlockManager heartbeats time out 
Makes it more clear what the threshold was for tuning spark.storage.blockManagerSlaveTimeoutMs

Before:
WARN  "Removing BlockManager BlockManagerId(201304022120-1976232532-5050-27464-0, myhostname, 51337) with no recent heart beats

After:
WARN  "Removing BlockManager BlockManagerId(201304022120-1976232532-5050-27464-0, myhostname, 51337) with no recent heart beats: 19216ms exceeds 15000ms
 2013-04-11 01:54:02 -03:00
Matei Zaharia c91ff8d493 Merge pull request #560 from ash211/patch-3 
Typos: cluser -> cluster
 2013-04-10 15:08:23 -07:00
Andrew Ash 6efc8cae8f Typos: cluser -> cluster 2013-04-10 13:44:10 -03:00
Prashant Sharma f31e41c270 Added class wrappers instead of object and incorporated most of matei comments 2013-04-10 15:08:01 +05:30
Prashant Sharma b67c638b85 Made shell loading synchronous as async loading confuses with out of order status messages 2013-04-10 14:48:49 +05:30
Prashant Sharma 9f26318bbd Fixed previously removed dependencies 2013-04-10 14:46:42 +05:30
Matei Zaharia 7cd83bf0f8 Merge pull request #559 from ash211/patch-example-whitespace 
Uniform whitespace across scala examples
 2013-04-09 22:07:35 -07:00
Matei Zaharia 271a4f3bb6 Merge pull request #555 from holdenk/master 
Retry failed ssh commands in the ec2 python script.
 2013-04-09 22:04:52 -07:00
Holden Karau 9bf24e1d61 Just use a loop for retries 2013-04-09 21:37:02 -07:00
Andrew Ash f1d8871ca1 Uniform whitespace across scala examples 2013-04-09 23:35:13 -04:00
Andrew Ash 18bd41d1a3 Don't pull in old versions of Jackson via hadoop-core 2013-04-09 14:44:47 -04:00
Matei Zaharia adba773fab Fix passing of superstep in Bagel to avoid seeing new values of the 
superstep value upon recomputation, and set the default storage level in
Bagel to MEMORY_AND_DISK
 2013-04-08 17:34:38 -04:00
Matei Zaharia eed54a25d8 Merge pull request #553 from pwendell/akka-standalone 
SPARK-724 - Have Akka logging enabled by default for standalone daemons
 2013-04-08 09:44:30 -07:00
Matei Zaharia 65caa8f711 Merge remote-tracking branch 'jey/bump-development-version-to-0.8.0' 
Conflicts:
	docs/_config.yml
	project/SparkBuild.scala
 2013-04-08 12:43:17 -04:00
Matei Zaharia 054feb6448 Fixed a bug with zip 2013-04-07 21:15:21 -04:00
Matei Zaharia b5900d47b1 Fix compile warning 2013-04-07 20:55:42 -04:00
Matei Zaharia 1cb3eb9762 Merge remote-tracking branch 'kalpit/master' 
Conflicts:
	project/SparkBuild.scala
 2013-04-07 20:54:18 -04:00
Patrick Wendell b496decf0a Updating based on code review 2013-04-07 17:44:48 -07:00
Matei Zaharia a1586412d6 Updated link to SBT 2013-04-07 20:31:19 -04:00
Matei Zaharia 6962d40b44 Fix deprecated warning 2013-04-07 20:27:33 -04:00
Matei Zaharia 34a47b8bc9 Update Scala version in docs 2013-04-07 20:27:03 -04:00
Matei Zaharia b362df39ea Merge pull request #552 from MLnick/master 
Bumping version for Twitter Algebird to latest
 2013-04-07 17:17:52 -07:00
Matei Zaharia 4b301902f5 Merge pull request #554 from andyk/scala2.9.3 
Fixes SPARK-723 - Update build to Scala 2.9.3
 2013-04-07 17:15:10 -07:00
Holden Karau ff2130a0ad Retry failed ssh commands. This is especially useful during system startup when the hosts may not have yet come on-line but can be useful at other points for people with flakey connections 2013-04-06 00:35:50 -07:00
Andy Konwinski 5555811bd5 Update build to Scala 2.9.3 2013-04-04 13:26:45 -07:00
Patrick Wendell 9b68ceaa26 SPARK-724 - Have Akka logging enabled by default for standalone daemons 
See the JIRA for more details.

I was only able to test the bash version (don't have Windows)
so maybe check over that the syntax is correct there.
 2013-04-03 14:29:46 -07:00
Nick Pentreath 0f54344fd8 Bumping Algebird version in examples now that it supports JDK 1.6 2013-04-03 13:15:34 +02:00
Matei Zaharia dfe98ca798 Merge pull request #550 from erikvanoosten/master 
corrected Algebird example
 2013-04-02 19:24:12 -07:00
Matei Zaharia b5d78307ca Merge pull request #551 from jey/python-bugfixes 
Python bugfixes
 2013-04-02 19:23:45 -07:00
Jey Kottalam 9a731f5a6d Fix Python saveAsTextFile doctest to not expect order to be preserved 2013-04-02 11:59:20 -07:00
Jey Kottalam 20604001e2 Fix argv handling in Python transitive closure example 2013-04-02 11:59:07 -07:00
Erik van oosten b5e60c3253 Corrected order of CountMinSketchMonoid arguments 2013-04-02 15:25:22 +03:00