ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2564 commits 6 branches 181 tags 397 MiB
Commit graph

2564 commits

This branch
This branch
All branches
Author SHA1 Message Date
Stephen Haberman a4adeb255c Merge branch 'master' into nomocks 
Conflicts:
	core/src/test/scala/spark/scheduler/DAGSchedulerSuite.scala
 2013-02-25 23:48:52 -06:00
Matei Zaharia 7e67c626ee Change version number to 0.7.0 2013-02-25 20:30:47 -08:00
Matei Zaharia c6380b1749 Merge branch 'master' of https://github.com/mesos/spark 2013-02-25 20:23:41 -08:00
Matei Zaharia 108cfaa5b4 Fix Windows script for finding examples JAR 2013-02-25 20:23:36 -08:00
Matei Zaharia 4223be3aa4 Merge pull request #503 from pwendell/bug-fix 
createNewSparkContext should use sparkHome/jars/environment.
 2013-02-25 19:43:05 -08:00
Patrick Wendell 284ba90958 createNewSparkContext should use sparkHome/jars/environment. 
This fixes a bug introduced by Matei's recent change.
 2013-02-25 19:40:52 -08:00
Matei Zaharia 5d7b591cfe Pass a code JAR to SparkContext in our examples. Fixes SPARK-594. 2013-02-25 19:34:32 -08:00
Matei Zaharia 7b88534932 Merge pull request #502 from tdas/master 
Very minor change in a testcase
 2013-02-25 18:11:00 -08:00
Tathagata Das bc4a6eb850 Changed Flume test to use the same port as other tests, so that can be controlled centrally. 2013-02-25 18:04:21 -08:00
Matei Zaharia c45d6f1248 Merge pull request #501 from tdas/master 
Fixed bug in BlockManager and added a testcase
 2013-02-25 17:35:31 -08:00
Tathagata Das c02e064938 Fixed replication bug in BlockManager 2013-02-25 17:27:46 -08:00
Matei Zaharia 4d480ec59e Fixed something that was reported as a compile error in ScalaDoc. 
For some reason, ScalaDoc complained about no such constructor for
StreamingContext; it doesn't seem like an actual Scala error but it
prevented sbt publish and from working because docs weren't built.
 2013-02-25 15:53:43 -08:00
Matei Zaharia 6494cab19d Update Hadoop dependency to 1.0.4 2013-02-25 15:38:21 -08:00
Tathagata Das 6a78ef0578 Merge pull request #500 from pwendell/streaming-docs 
Minor changes based on feedback
 2013-02-25 15:28:45 -08:00
Patrick Wendell 8316534eef meta-data 2013-02-25 15:27:04 -08:00
Patrick Wendell 918ee25867 One more change done with TD 2013-02-25 15:24:17 -08:00
Matei Zaharia 351ac5233e Some tweaks to docs 2013-02-25 15:19:05 -08:00
Matei Zaharia 2ae15353a1 Merge branch 'master' of github.com:mesos/spark 2013-02-25 15:14:16 -08:00
Matei Zaharia 490f056cdd Allow passing sparkHome and JARs to StreamingContext constructor 
Also warns if spark.cleaner.ttl is not set in the version where you pass
your own SparkContext.
 2013-02-25 15:13:30 -08:00
Patrick Wendell 07f2618769 Minor changes based on feedback 2013-02-25 15:09:59 -08:00
Matei Zaharia 271a40f0e4 Merge pull request #499 from pwendell/streaming-docs 
Some changes to streaming failure docs.
 2013-02-25 14:47:32 -08:00
Patrick Wendell 50ce0516e6 Some changes to streaming failure docs. 
TD gave me the go-ahead to just make these changes:
- Define stateful dstream
- Some minor wording fixes
 2013-02-25 14:38:39 -08:00
Matei Zaharia 568bdaf8ae Set spark.deploy.spreadOut to true by default in 0.7 (improves locality) 2013-02-25 14:34:55 -08:00
Matei Zaharia 5d4a0ac794 Some tweaks to docs 2013-02-25 14:23:03 -08:00
Matei Zaharia 1ef58dadcc Add a config property for Akka lifecycle event logging 2013-02-25 14:01:24 -08:00
Matei Zaharia 6b87ef7c86 Fix compile error 2013-02-25 14:01:16 -08:00
Matei Zaharia 01bd136ba5 Use public method sparkContext instead of protected sc in streaming examples 2013-02-25 13:27:11 -08:00
Matei Zaharia 848321f910 Change doc color scheme slightly for Spark 0.7 (to differ from 0.6) 2013-02-25 13:15:30 -08:00
Matei Zaharia 3c7dcb61ab Use a single setting for disabling API doc build 2013-02-25 13:15:12 -08:00
Matei Zaharia 0cb8ee93cd Merge branch 'master' of github.com:mesos/spark 2013-02-25 13:10:35 -08:00
Matei Zaharia ceaec4a675 Merge pull request #498 from pwendell/shutup-akka 
Disable remote lifecycle logging from Akka.
 2013-02-25 12:31:24 -08:00
Patrick Wendell 85a85646d9 Disable remote lifecycle logging from Akka. 
This changes the default setting to `off` for remote lifecycle events. When this is on, it is very chatty at the INFO level. It also prints out several ERROR messages sometimes when sc.stop() is called.
 2013-02-25 12:25:43 -08:00
Matei Zaharia 25f737804a Change tabs to spaces 2013-02-25 11:53:55 -08:00
Imran Rashid 8f17387d97 remove bogus comment 2013-02-25 10:31:06 -08:00
Matei Zaharia 6ae9a22c3e Get spark.default.paralellism on each call to defaultPartitioner, 
instead of only once, in case the user changes it across Spark uses
 2013-02-25 10:28:08 -08:00
Matei Zaharia d6e6abece3 Merge pull request #459 from stephenh/bettersplits 
Change defaultPartitioner to use upstream split size.
 2013-02-25 09:22:04 -08:00
Matei Zaharia fb76250598 Merge pull request #497 from ScrapCodes/dep-resolution-fix 
Moving akka dependency resolver to shared.
 2013-02-25 00:16:52 -08:00
Prashant Sharma 254acb1666 Moving akka dependency resolver to shared. 2013-02-25 13:37:07 +05:30
Stephen Haberman c44ccf2862 Use default parallelism if its set. 2013-02-24 23:54:03 -06:00
Stephen Haberman 44032bc476 Merge branch 'master' into bettersplits 
Conflicts:
	core/src/main/scala/spark/RDD.scala
	core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala
	core/src/test/scala/spark/ShuffleSuite.scala
 2013-02-24 22:08:14 -06:00
Matei Zaharia 22f68b7e6b Merge pull request #494 from mesos/streaming 
Final set of Spark Streaming changes
 2013-02-24 17:48:17 -08:00
Tathagata Das f9881b10c6 Merge pull request #495 from tdas/streaming 
Changed class paths and dependencies based on Matei's comments
 2013-02-24 16:39:04 -08:00
Tathagata Das abb5471865 Removing duplicate doc. 2013-02-24 16:32:44 -08:00
Tathagata Das 5ab37be983 Fixed class paths and dependencies based on Matei's comments. 2013-02-24 16:24:52 -08:00
Tathagata Das 32f1a2d540 Merge pull request #493 from tdas/streaming 
Streaming bug fixes
 2013-02-24 14:46:19 -08:00
Matei Zaharia 066ce0ffa4 Merge pull request #492 from JoshRosen/pyspark-split-rename 
Change numSplits to numPartitions in PySpark
 2013-02-24 14:11:07 -08:00
Josh Rosen 2c966c98fb Change numSplits to numPartitions in PySpark. 2013-02-24 13:25:09 -08:00
Tathagata Das 28f8b721f6 Added back the initial spark job before starting streaming receivers 2013-02-24 13:01:54 -08:00
Tathagata Das f282bc4960 Changed Algebird from 0.1.9 to 0.1.8 2013-02-24 12:44:12 -08:00
Tathagata Das dff53d1b94 Merge branch 'mesos-master' into streaming 2013-02-24 12:17:22 -08:00