Mridul Muralidharan
a402b23bcd
Fudge order of classpath - so that our jars take precedence over what is in CLASSPATH variable. Sounds logical, hope there is no issue cos of it
2013-04-17 05:52:00 +05:30
seanm
7e56e99573
Surfacing decoders on KafkaInputDStream
2013-04-16 17:17:16 -06:00
Mridul Muralidharan
bcdde331c3
Move from master to driver
2013-04-17 04:12:18 +05:30
Joseph E. Gonzalez
a8dad98c55
merged with trunk
2013-04-16 11:39:02 -07:00
Joseph E. Gonzalez
2635416cee
switching from floats to doubles in pagerank and sssp
2013-04-16 11:27:56 -07:00
seanm
ab0f834dbb
adding spark.streaming.blockInterval property
2013-04-16 11:57:05 -06:00
Jey Kottalam
6bfe4bf3eb
Increase ReservedCodeCacheSize for sbt
2013-04-16 09:50:59 -07:00
Mridul Muralidharan
ad80f68eb5
remove spurious debug statements
2013-04-16 22:15:34 +05:30
Mridul Muralidharan
f7969f72ee
Fix exception when checkpoint path does not exist (no data in rdd which is being checkpointed for example)
2013-04-16 21:51:38 +05:30
Mridul Muralidharan
323ab8ff3b
Scala does not prevent variable shadowing ! Sick error due to it ...
2013-04-16 17:05:10 +05:30
Prashant Sharma
36ccb35371
Changed spark context as lazy val to become just val.
2013-04-16 11:13:51 +05:30
shane-huang
b493f55a4f
fix a bug in netty Block Fetcher
...
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
2013-04-16 10:01:01 +08:00
Mridul Muralidharan
59c380d69a
Fix npe
2013-04-16 03:29:38 +05:30
Mridul Muralidharan
dd2b64ec97
Fix bug with atomic update
2013-04-16 03:19:24 +05:30
Mridul Muralidharan
5540ab8243
Use hostname instead of hostport for executor, fix creation of workdir
2013-04-16 02:57:43 +05:30
Mridul Muralidharan
eb7e95e833
Commit job to persist files
2013-04-16 02:56:36 +05:30
Matei Zaharia
a64c107449
Make ShuffledRDD.prev transient
2013-04-15 16:41:51 -04:00
seanm
b42d68c8ce
fixing Spark Streaming count() so that 0 will be emitted when there is nothing to count
2013-04-15 12:54:55 -06:00
Prashant Sharma
19b0256ae4
Added standalone cluster repl suite
2013-04-15 19:49:40 +05:30
Mridul Muralidharan
19652a44be
Fix issue with FileSuite failing
2013-04-15 19:16:36 +05:30
Mridul Muralidharan
54b3d45b81
Checkpoint commit - compiles and passes a lot of tests - not all though, looking into FileSuite issues
2013-04-15 18:26:50 +05:30
Mridul Muralidharan
d90d2af103
Checkpoint commit - compiles and passes a lot of tests - not all though, looking into FileSuite issues
2013-04-15 18:12:11 +05:30
Matei Zaharia
ec5e553b41
Merge pull request #558 from ash211/patch-jackson-conflict
...
Don't pull in old versions of Jackson via hadoop-core
2013-04-14 08:20:13 -07:00
Matei Zaharia
c1c219e263
Merge pull request #564 from maspotts/master
...
Allow latest scala in PATH, with SCALA_HOME as override (instead of vice-versa)
2013-04-14 08:11:23 -07:00
Matei Zaharia
c35d530bcf
Fix compile error
2013-04-13 12:43:12 -04:00
Matei Zaharia
7c10b3e3cd
Merge pull request #565 from andyk/master
...
Update wording of section on RDD operations in quick start guide in docs
2013-04-12 20:55:22 -07:00
Andy Konwinski
60a91b3b59
Update quick-start.md heading on Operations (not just Transformations).
2013-04-12 12:34:51 -07:00
Mike
6f68860891
Reversed the order of tests to find a scala executable (in the case when SPARK_LAUNCH_WITH_SCALA is defined): instead of checking in the PATH first, and only then (if not found) for SCALA_HOME, now we check for SCALA_HOME first, and only then (if not defined) do we look in the PATH. The advantage is that now if the user has a more recent (non-compatible) version of scala in her PATH, she can use SCALA_HOME to point to the older (compatible) version for use with spark.
...
Suggested by Josh Rosen in this thread:
https://groups.google.com/forum/?fromgroups=#!topic/spark-users/NC9JKvP8808
2013-04-11 20:52:06 -07:00
Matei Zaharia
077ae0a197
Merge pull request #561 from ash211/patch-4
...
Add details when BlockManager heartbeats time out
2013-04-11 19:34:14 -07:00
Matei Zaharia
e2aa87558d
Merge branch 'master' of github.com:mesos/spark
2013-04-11 22:30:05 -04:00
Matei Zaharia
ed336e0d44
Fix tests from different projects running in parallel in SBT 0.12
2013-04-11 22:29:37 -04:00
Andrew Ash
29d3440efb
Add details when BlockManager heartbeats time out
...
Makes it more clear what the threshold was for tuning spark.storage.blockManagerSlaveTimeoutMs
Before:
WARN "Removing BlockManager BlockManagerId(201304022120-1976232532-5050-27464-0, myhostname, 51337) with no recent heart beats
After:
WARN "Removing BlockManager BlockManagerId(201304022120-1976232532-5050-27464-0, myhostname, 51337) with no recent heart beats: 19216ms exceeds 15000ms
2013-04-11 01:54:02 -03:00
Matei Zaharia
c91ff8d493
Merge pull request #560 from ash211/patch-3
...
Typos: cluser -> cluster
2013-04-10 15:08:23 -07:00
Andrew Ash
6efc8cae8f
Typos: cluser -> cluster
2013-04-10 13:44:10 -03:00
Prashant Sharma
f31e41c270
Added class wrappers instead of object and incorporated most of matei comments
2013-04-10 15:08:01 +05:30
Prashant Sharma
b67c638b85
Made shell loading synchronous as async loading confuses with out of order status messages
2013-04-10 14:48:49 +05:30
Prashant Sharma
9f26318bbd
Fixed previously removed dependencies
2013-04-10 14:46:42 +05:30
Matei Zaharia
7cd83bf0f8
Merge pull request #559 from ash211/patch-example-whitespace
...
Uniform whitespace across scala examples
2013-04-09 22:07:35 -07:00
Matei Zaharia
271a4f3bb6
Merge pull request #555 from holdenk/master
...
Retry failed ssh commands in the ec2 python script.
2013-04-09 22:04:52 -07:00
Holden Karau
9bf24e1d61
Just use a loop for retries
2013-04-09 21:37:02 -07:00
Andrew Ash
f1d8871ca1
Uniform whitespace across scala examples
2013-04-09 23:35:13 -04:00
Matei Zaharia
8ac9efba5a
Merge pull request #527 from Reinvigorate/sm-kafka-cleanup
...
KafkaInputDStream fixes and improvements
2013-04-09 13:50:50 -07:00
Andrew Ash
18bd41d1a3
Don't pull in old versions of Jackson via hadoop-core
2013-04-09 14:44:47 -04:00
Andrew xia
2f883c515f
Contiue to update codes for scala code style
...
1.refactor braces for "class" "if" "while" "for" "match"
2.make code lines less than 100
3.refactor class parameter and extends defination
2013-04-09 13:02:50 +08:00
Matei Zaharia
adba773fab
Fix passing of superstep in Bagel to avoid seeing new values of the
...
superstep value upon recomputation, and set the default storage level in
Bagel to MEMORY_AND_DISK
2013-04-08 17:34:38 -04:00
Matei Zaharia
eed54a25d8
Merge pull request #553 from pwendell/akka-standalone
...
SPARK-724 - Have Akka logging enabled by default for standalone daemons
2013-04-08 09:44:30 -07:00
Matei Zaharia
65caa8f711
Merge remote-tracking branch 'jey/bump-development-version-to-0.8.0'
...
Conflicts:
docs/_config.yml
project/SparkBuild.scala
2013-04-08 12:43:17 -04:00
Matei Zaharia
054feb6448
Fixed a bug with zip
2013-04-07 21:15:21 -04:00
Matei Zaharia
b5900d47b1
Fix compile warning
2013-04-07 20:55:42 -04:00
Matei Zaharia
1cb3eb9762
Merge remote-tracking branch 'kalpit/master'
...
Conflicts:
project/SparkBuild.scala
2013-04-07 20:54:18 -04:00