ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
4973 commits 6 branches 181 tags 397 MiB
Commit graph

4973 commits

This branch
This branch
All branches
Author SHA1 Message Date
Mridul Muralidharan 46779b4745 Move back to 2.0.2-alpha, since 2.0.3-alpha is not available in cloudera yet 2013-04-17 05:53:28 +05:30
Mridul Muralidharan 02dffd2eb0 Ensure all ask/await block for spark.akka.askTimeout - so that it is controllable : instead of arbitrary timeouts spread across codebase. In our tests, we use 30 seconds, though default of 10 is maintained 2013-04-17 05:52:57 +05:30
Mridul Muralidharan a402b23bcd Fudge order of classpath - so that our jars take precedence over what is in CLASSPATH variable. Sounds logical, hope there is no issue cos of it 2013-04-17 05:52:00 +05:30
seanm 7e56e99573 Surfacing decoders on KafkaInputDStream 2013-04-16 17:17:16 -06:00
Mridul Muralidharan bcdde331c3 Move from master to driver 2013-04-17 04:12:18 +05:30
Joseph E. Gonzalez a8dad98c55 merged with trunk 2013-04-16 11:39:02 -07:00
Joseph E. Gonzalez 2635416cee switching from floats to doubles in pagerank and sssp 2013-04-16 11:27:56 -07:00
seanm ab0f834dbb adding spark.streaming.blockInterval property 2013-04-16 11:57:05 -06:00
Jey Kottalam 6bfe4bf3eb Increase ReservedCodeCacheSize for sbt 2013-04-16 09:50:59 -07:00
Mridul Muralidharan ad80f68eb5 remove spurious debug statements 2013-04-16 22:15:34 +05:30
Mridul Muralidharan f7969f72ee Fix exception when checkpoint path does not exist (no data in rdd which is being checkpointed for example) 2013-04-16 21:51:38 +05:30
Mridul Muralidharan 323ab8ff3b Scala does not prevent variable shadowing ! Sick error due to it ... 2013-04-16 17:05:10 +05:30
shane-huang b493f55a4f fix a bug in netty Block Fetcher 
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
 2013-04-16 10:01:01 +08:00
Mridul Muralidharan 59c380d69a Fix npe 2013-04-16 03:29:38 +05:30
Mridul Muralidharan dd2b64ec97 Fix bug with atomic update 2013-04-16 03:19:24 +05:30
Mridul Muralidharan 5540ab8243 Use hostname instead of hostport for executor, fix creation of workdir 2013-04-16 02:57:43 +05:30
Mridul Muralidharan eb7e95e833 Commit job to persist files 2013-04-16 02:56:36 +05:30
Matei Zaharia a64c107449 Make ShuffledRDD.prev transient 2013-04-15 16:41:51 -04:00
seanm b42d68c8ce fixing Spark Streaming count() so that 0 will be emitted when there is nothing to count 2013-04-15 12:54:55 -06:00
Mridul Muralidharan 19652a44be Fix issue with FileSuite failing 2013-04-15 19:16:36 +05:30
Mridul Muralidharan 54b3d45b81 Checkpoint commit - compiles and passes a lot of tests - not all though, looking into FileSuite issues 2013-04-15 18:26:50 +05:30
Mridul Muralidharan d90d2af103 Checkpoint commit - compiles and passes a lot of tests - not all though, looking into FileSuite issues 2013-04-15 18:12:11 +05:30
Matei Zaharia ec5e553b41 Merge pull request #558 from ash211/patch-jackson-conflict 
Don't pull in old versions of Jackson via hadoop-core
 2013-04-14 08:20:13 -07:00
Matei Zaharia c1c219e263 Merge pull request #564 from maspotts/master 
Allow latest scala in PATH, with SCALA_HOME as override (instead of vice-versa)
 2013-04-14 08:11:23 -07:00
Matei Zaharia c35d530bcf Fix compile error 2013-04-13 12:43:12 -04:00
Matei Zaharia 7c10b3e3cd Merge pull request #565 from andyk/master 
Update wording of section on RDD operations in quick start guide in docs
 2013-04-12 20:55:22 -07:00
Andy Konwinski 60a91b3b59 Update quick-start.md heading on Operations (not just Transformations). 2013-04-12 12:34:51 -07:00
Mike 6f68860891 Reversed the order of tests to find a scala executable (in the case when SPARK_LAUNCH_WITH_SCALA is defined): instead of checking in the PATH first, and only then (if not found) for SCALA_HOME, now we check for SCALA_HOME first, and only then (if not defined) do we look in the PATH. The advantage is that now if the user has a more recent (non-compatible) version of scala in her PATH, she can use SCALA_HOME to point to the older (compatible) version for use with spark. 
Suggested by Josh Rosen in this thread:

  https://groups.google.com/forum/?fromgroups=#!topic/spark-users/NC9JKvP8808
 2013-04-11 20:52:06 -07:00
Matei Zaharia 077ae0a197 Merge pull request #561 from ash211/patch-4 
Add details when BlockManager heartbeats time out
 2013-04-11 19:34:14 -07:00
Matei Zaharia e2aa87558d Merge branch 'master' of github.com:mesos/spark 2013-04-11 22:30:05 -04:00
Matei Zaharia ed336e0d44 Fix tests from different projects running in parallel in SBT 0.12 2013-04-11 22:29:37 -04:00
Andrew Ash 29d3440efb Add details when BlockManager heartbeats time out 
Makes it more clear what the threshold was for tuning spark.storage.blockManagerSlaveTimeoutMs

Before:
WARN  "Removing BlockManager BlockManagerId(201304022120-1976232532-5050-27464-0, myhostname, 51337) with no recent heart beats

After:
WARN  "Removing BlockManager BlockManagerId(201304022120-1976232532-5050-27464-0, myhostname, 51337) with no recent heart beats: 19216ms exceeds 15000ms
 2013-04-11 01:54:02 -03:00
Matei Zaharia c91ff8d493 Merge pull request #560 from ash211/patch-3 
Typos: cluser -> cluster
 2013-04-10 15:08:23 -07:00
Andrew Ash 6efc8cae8f Typos: cluser -> cluster 2013-04-10 13:44:10 -03:00
Matei Zaharia 7cd83bf0f8 Merge pull request #559 from ash211/patch-example-whitespace 
Uniform whitespace across scala examples
 2013-04-09 22:07:35 -07:00
Matei Zaharia 271a4f3bb6 Merge pull request #555 from holdenk/master 
Retry failed ssh commands in the ec2 python script.
 2013-04-09 22:04:52 -07:00
Holden Karau 9bf24e1d61 Just use a loop for retries 2013-04-09 21:37:02 -07:00
Andrew Ash f1d8871ca1 Uniform whitespace across scala examples 2013-04-09 23:35:13 -04:00
Matei Zaharia 8ac9efba5a Merge pull request #527 from Reinvigorate/sm-kafka-cleanup 
KafkaInputDStream fixes and improvements
 2013-04-09 13:50:50 -07:00
Andrew Ash 18bd41d1a3 Don't pull in old versions of Jackson via hadoop-core 2013-04-09 14:44:47 -04:00
Andrew xia 2f883c515f Contiue to update codes for scala code style 
1.refactor braces for "class" "if" "while" "for" "match"
2.make code lines less than 100
3.refactor class parameter and extends defination
 2013-04-09 13:02:50 +08:00
Matei Zaharia adba773fab Fix passing of superstep in Bagel to avoid seeing new values of the 
superstep value upon recomputation, and set the default storage level in
Bagel to MEMORY_AND_DISK
 2013-04-08 17:34:38 -04:00
Matei Zaharia eed54a25d8 Merge pull request #553 from pwendell/akka-standalone 
SPARK-724 - Have Akka logging enabled by default for standalone daemons
 2013-04-08 09:44:30 -07:00
Matei Zaharia 65caa8f711 Merge remote-tracking branch 'jey/bump-development-version-to-0.8.0' 
Conflicts:
	docs/_config.yml
	project/SparkBuild.scala
 2013-04-08 12:43:17 -04:00
Matei Zaharia 054feb6448 Fixed a bug with zip 2013-04-07 21:15:21 -04:00
Matei Zaharia b5900d47b1 Fix compile warning 2013-04-07 20:55:42 -04:00
Matei Zaharia 1cb3eb9762 Merge remote-tracking branch 'kalpit/master' 
Conflicts:
	project/SparkBuild.scala
 2013-04-07 20:54:18 -04:00
Patrick Wendell b496decf0a Updating based on code review 2013-04-07 17:44:48 -07:00
Matei Zaharia a1586412d6 Updated link to SBT 2013-04-07 20:31:19 -04:00
Matei Zaharia 6962d40b44 Fix deprecated warning 2013-04-07 20:27:33 -04:00