Stephen Haberman
80eecd2cb1
Make Executor fields volatile since they're read from the thread pool.
2013-02-28 10:41:07 -06:00
Mosharaf Chowdhury
4ab387bcdb
Fixed master datastructure updates after removing an application; and a typo.
2013-02-27 13:52:44 -08:00
Matei Zaharia
db9b90fdbd
Change version to 0.7.1-SNAPSHOT for development branch
2013-02-27 09:15:26 -08:00
Matei Zaharia
4f840f4e54
Added commented-out Google analytics code for website docs
2013-02-27 09:14:11 -08:00
Matei Zaharia
baa30fcd99
Use new Spark EC2 scripts by default
2013-02-26 23:38:50 -08:00
Matei Zaharia
fadeb1ddea
More doc tweaks
2013-02-26 23:20:49 -08:00
Matei Zaharia
22334eafd9
Some tweaks to docs
2013-02-26 22:52:38 -08:00
Matei Zaharia
434a1ce773
Small hack to work around multiple JARs being built by sbt package
2013-02-26 12:24:18 -08:00
Matei Zaharia
ece3edfffa
Fix a problem with no hosts being counted as alive in the first job
2013-02-26 12:11:03 -08:00
Matei Zaharia
73697e2891
Fix overly large thread names in PySpark
2013-02-26 12:07:59 -08:00
Stephen Haberman
db957e5bd7
Fix MapOutputTrackerSuite.
2013-02-26 01:38:50 -06:00
Matei Zaharia
9a046e30ac
Switch docs to use Akka repo instead of Typesafe
2013-02-25 22:18:47 -08:00
Stephen Haberman
a65aa549ff
Override DAGScheduler.runLocally so we can remove the Thread.sleep.
2013-02-25 23:49:32 -06:00
Stephen Haberman
a4adeb255c
Merge branch 'master' into nomocks
...
Conflicts:
core/src/test/scala/spark/scheduler/DAGSchedulerSuite.scala
2013-02-25 23:48:52 -06:00
Matei Zaharia
7e67c626ee
Change version number to 0.7.0
2013-02-25 20:30:47 -08:00
Matei Zaharia
c6380b1749
Merge branch 'master' of https://github.com/mesos/spark
2013-02-25 20:23:41 -08:00
Matei Zaharia
108cfaa5b4
Fix Windows script for finding examples JAR
2013-02-25 20:23:36 -08:00
Matei Zaharia
4223be3aa4
Merge pull request #503 from pwendell/bug-fix
...
createNewSparkContext should use sparkHome/jars/environment.
2013-02-25 19:43:05 -08:00
Patrick Wendell
284ba90958
createNewSparkContext should use sparkHome/jars/environment.
...
This fixes a bug introduced by Matei's recent change.
2013-02-25 19:40:52 -08:00
Matei Zaharia
5d7b591cfe
Pass a code JAR to SparkContext in our examples. Fixes SPARK-594.
2013-02-25 19:34:32 -08:00
Matei Zaharia
7b88534932
Merge pull request #502 from tdas/master
...
Very minor change in a testcase
2013-02-25 18:11:00 -08:00
Tathagata Das
bc4a6eb850
Changed Flume test to use the same port as other tests, so that can be controlled centrally.
2013-02-25 18:04:21 -08:00
Matei Zaharia
c45d6f1248
Merge pull request #501 from tdas/master
...
Fixed bug in BlockManager and added a testcase
2013-02-25 17:35:31 -08:00
Tathagata Das
c02e064938
Fixed replication bug in BlockManager
2013-02-25 17:27:46 -08:00
Matei Zaharia
4d480ec59e
Fixed something that was reported as a compile error in ScalaDoc.
...
For some reason, ScalaDoc complained about no such constructor for
StreamingContext; it doesn't seem like an actual Scala error but it
prevented sbt publish and from working because docs weren't built.
2013-02-25 15:53:43 -08:00
Matei Zaharia
6494cab19d
Update Hadoop dependency to 1.0.4
2013-02-25 15:38:21 -08:00
Tathagata Das
6a78ef0578
Merge pull request #500 from pwendell/streaming-docs
...
Minor changes based on feedback
2013-02-25 15:28:45 -08:00
Patrick Wendell
8316534eef
meta-data
2013-02-25 15:27:04 -08:00
Patrick Wendell
918ee25867
One more change done with TD
2013-02-25 15:24:17 -08:00
Matei Zaharia
351ac5233e
Some tweaks to docs
2013-02-25 15:19:05 -08:00
Matei Zaharia
2ae15353a1
Merge branch 'master' of github.com:mesos/spark
2013-02-25 15:14:16 -08:00
Matei Zaharia
490f056cdd
Allow passing sparkHome and JARs to StreamingContext constructor
...
Also warns if spark.cleaner.ttl is not set in the version where you pass
your own SparkContext.
2013-02-25 15:13:30 -08:00
Patrick Wendell
07f2618769
Minor changes based on feedback
2013-02-25 15:09:59 -08:00
Matei Zaharia
271a40f0e4
Merge pull request #499 from pwendell/streaming-docs
...
Some changes to streaming failure docs.
2013-02-25 14:47:32 -08:00
Patrick Wendell
50ce0516e6
Some changes to streaming failure docs.
...
TD gave me the go-ahead to just make these changes:
- Define stateful dstream
- Some minor wording fixes
2013-02-25 14:38:39 -08:00
Matei Zaharia
568bdaf8ae
Set spark.deploy.spreadOut to true by default in 0.7 (improves locality)
2013-02-25 14:34:55 -08:00
Matei Zaharia
5d4a0ac794
Some tweaks to docs
2013-02-25 14:23:03 -08:00
Matei Zaharia
1ef58dadcc
Add a config property for Akka lifecycle event logging
2013-02-25 14:01:24 -08:00
Matei Zaharia
6b87ef7c86
Fix compile error
2013-02-25 14:01:16 -08:00
Matei Zaharia
01bd136ba5
Use public method sparkContext instead of protected sc in streaming examples
2013-02-25 13:27:11 -08:00
Matei Zaharia
848321f910
Change doc color scheme slightly for Spark 0.7 (to differ from 0.6)
2013-02-25 13:15:30 -08:00
Matei Zaharia
3c7dcb61ab
Use a single setting for disabling API doc build
2013-02-25 13:15:12 -08:00
Matei Zaharia
0cb8ee93cd
Merge branch 'master' of github.com:mesos/spark
2013-02-25 13:10:35 -08:00
Matei Zaharia
ceaec4a675
Merge pull request #498 from pwendell/shutup-akka
...
Disable remote lifecycle logging from Akka.
2013-02-25 12:31:24 -08:00
Patrick Wendell
85a85646d9
Disable remote lifecycle logging from Akka.
...
This changes the default setting to `off` for remote lifecycle events. When this is on, it is very chatty at the INFO level. It also prints out several ERROR messages sometimes when sc.stop() is called.
2013-02-25 12:25:43 -08:00
Matei Zaharia
25f737804a
Change tabs to spaces
2013-02-25 11:53:55 -08:00
Imran Rashid
8f17387d97
remove bogus comment
2013-02-25 10:31:06 -08:00
Matei Zaharia
6ae9a22c3e
Get spark.default.paralellism on each call to defaultPartitioner,
...
instead of only once, in case the user changes it across Spark uses
2013-02-25 10:28:08 -08:00
Matei Zaharia
d6e6abece3
Merge pull request #459 from stephenh/bettersplits
...
Change defaultPartitioner to use upstream split size.
2013-02-25 09:22:04 -08:00
Matei Zaharia
fb76250598
Merge pull request #497 from ScrapCodes/dep-resolution-fix
...
Moving akka dependency resolver to shared.
2013-02-25 00:16:52 -08:00