ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2454 commits 6 branches 181 tags 397 MiB
Commit graph

2454 commits

This branch
This branch
All branches
Author SHA1 Message Date
Nick Pentreath 568ddf7330 Adding Java K-Means example 2013-03-19 15:29:22 +02:00
Nick Pentreath b990caeb80 Changes to more closely match line length limit style 2013-03-17 20:03:27 +02:00
Nick Pentreath 13757b1198 Adding Java versions of Pi and LogQuery 2013-03-15 10:52:01 +02:00
Matei Zaharia cbf8f0d4dd Merge pull request #513 from MLnick/bagel-caching 
Adds choice of persistence level to Bagel.
 2013-03-11 00:23:57 -07:00
Nick Pentreath 8dd943fc3e Fix doc style 2013-03-11 09:14:00 +02:00
Matei Zaharia 2e1bbc4e7e Merge remote-tracking branch 'woggling/dag-sched-driver-port' 
Conflicts:
	core/src/test/scala/spark/scheduler/DAGSchedulerSuite.scala
 2013-03-10 16:52:54 -07:00
Matei Zaharia d4e29ea878 Update kryo-serializers version in pom.xml to match previous commit 2013-03-10 15:49:11 -07:00
Matei Zaharia 91a9d093bd Merge pull request #512 from patelh/fix-kryo-serializer 
Fix reference bug in Kryo serializer, add test, update version
 2013-03-10 15:48:23 -07:00
Matei Zaharia 557cfd0f4d Merge pull request #515 from woggling/deploy-app-death 
Notify standalone deploy client of application death.
 2013-03-10 15:44:57 -07:00
Matei Zaharia a59cc6060f Merge remote-tracking branch 'stephenh/nomocks' 
Conflicts:
	core/src/main/scala/spark/storage/BlockManagerMaster.scala
	core/src/test/scala/spark/scheduler/DAGSchedulerSuite.scala
 2013-03-10 13:39:10 -07:00
Charles Reiss b0983c5762 Notify standalone deploy client of application death. 
Usually, this isn't necessary since the application will be removed
as a result of the deploy client disconnecting, but occassionally, the
standalone deploy master removes an application otherwise.

Also mark applications as FAILED instead of FINISHED when they are
killed as a result of their executors failing too many times.
 2013-03-09 11:29:45 -08:00
Charles Reiss d0216cb38b Prevent DAGSchedulerSuite from corrupting driver.port. 
Use the LocalSparkContext abstraction to properly manage clearing
spark.driver.port.
 2013-03-09 10:49:02 -08:00
Nick Pentreath d35c5a5176 Adding test for non-default persistence level 2013-03-09 12:52:16 +02:00
Nick Pentreath 1e981d8b26 Added choice of persitance level to Bagel. Also added documentation. 2013-03-09 12:40:48 +02:00
Hiral Patel 664e5fd24b Fix reference bug in Kryo serializer, add test, update version 2013-03-07 22:16:11 -08:00
Matei Zaharia 9f0dc829cb Fix TaskMetrics not being serializable 2013-03-04 12:08:31 -08:00
Matei Zaharia 04fb81ffe5 Merge pull request #506 from rxin/spark-706 
Fixed SPARK-706: Failures in block manager put leads to read task hanging.
 2013-03-03 17:20:07 -08:00
Matei Zaharia 6cf4be4b39 Merge pull request #462 from squito/stageInfo 
Track assorted metrics for each task, report summaries to user at stage completion
 2013-03-03 17:16:22 -08:00
Imran Rashid 0bd1d00c2a minor cleanup based on feedback in review request 2013-03-03 16:46:45 -08:00
Imran Rashid f1006b99ff change CleanupIterator to CompletionIterator 2013-03-03 16:39:05 -08:00
Imran Rashid 8fef5b9c5f refactoring of TaskMetrics 2013-03-03 16:34:04 -08:00
Imran Rashid d36abdb053 Merge branch 'master' into stageInfo 2013-03-03 15:20:46 -08:00
Matei Zaharia 6bfc7cad6b Merge pull request #504 from mosharaf/master 
Worker address was getting removed when removing an app.
 2013-03-02 22:14:49 -08:00
Matei Zaharia 94b3db136d Merge pull request #508 from markhamstra/TestServerInUse 
Avoid bind failure in InputStreamsSuite
 2013-03-02 22:13:52 -08:00
Mark Hamstra b409073102 Instead of failing to bind to a fixed, already-in-use port, let the OS choose an available port for TestServer. 2013-03-01 15:05:07 -08:00
Matei Zaharia 25c71d3e12 Merge pull request #507 from markhamstra/poms271 
bump version to 0.7.1-SNAPSHOT in the subproject poms
 2013-03-01 08:00:18 -08:00
Mark Hamstra 8b06b359da bump version to 0.7.1-SNAPSHOT in the subproject poms to keep the maven build building. 2013-02-28 23:34:34 -08:00
Reynold Xin 44134e12bb Fixed SPARK-706: Failures in block manager put leads to read task 
hanging.
 2013-02-28 15:14:59 -08:00
Mosharaf Chowdhury 4ab387bcdb Fixed master datastructure updates after removing an application; and a typo. 2013-02-27 13:52:44 -08:00
Matei Zaharia db9b90fdbd Change version to 0.7.1-SNAPSHOT for development branch 2013-02-27 09:15:26 -08:00
Matei Zaharia 4f840f4e54 Added commented-out Google analytics code for website docs 2013-02-27 09:14:11 -08:00
Matei Zaharia baa30fcd99 Use new Spark EC2 scripts by default 2013-02-26 23:38:50 -08:00
Matei Zaharia fadeb1ddea More doc tweaks 2013-02-26 23:20:49 -08:00
Matei Zaharia 22334eafd9 Some tweaks to docs 2013-02-26 22:52:38 -08:00
Matei Zaharia 434a1ce773 Small hack to work around multiple JARs being built by sbt package 2013-02-26 12:24:18 -08:00
Matei Zaharia ece3edfffa Fix a problem with no hosts being counted as alive in the first job 2013-02-26 12:11:03 -08:00
Matei Zaharia 73697e2891 Fix overly large thread names in PySpark 2013-02-26 12:07:59 -08:00
Stephen Haberman db957e5bd7 Fix MapOutputTrackerSuite. 2013-02-26 01:38:50 -06:00
Matei Zaharia 9a046e30ac Switch docs to use Akka repo instead of Typesafe 2013-02-25 22:18:47 -08:00
Stephen Haberman a65aa549ff Override DAGScheduler.runLocally so we can remove the Thread.sleep. 2013-02-25 23:49:32 -06:00
Stephen Haberman a4adeb255c Merge branch 'master' into nomocks 
Conflicts:
	core/src/test/scala/spark/scheduler/DAGSchedulerSuite.scala
 2013-02-25 23:48:52 -06:00
Matei Zaharia 7e67c626ee Change version number to 0.7.0 2013-02-25 20:30:47 -08:00
Matei Zaharia c6380b1749 Merge branch 'master' of https://github.com/mesos/spark 2013-02-25 20:23:41 -08:00
Matei Zaharia 108cfaa5b4 Fix Windows script for finding examples JAR 2013-02-25 20:23:36 -08:00
Matei Zaharia 4223be3aa4 Merge pull request #503 from pwendell/bug-fix 
createNewSparkContext should use sparkHome/jars/environment.
 2013-02-25 19:43:05 -08:00
Patrick Wendell 284ba90958 createNewSparkContext should use sparkHome/jars/environment. 
This fixes a bug introduced by Matei's recent change.
 2013-02-25 19:40:52 -08:00
Matei Zaharia 5d7b591cfe Pass a code JAR to SparkContext in our examples. Fixes SPARK-594. 2013-02-25 19:34:32 -08:00
Matei Zaharia 7b88534932 Merge pull request #502 from tdas/master 
Very minor change in a testcase
 2013-02-25 18:11:00 -08:00
Tathagata Das bc4a6eb850 Changed Flume test to use the same port as other tests, so that can be controlled centrally. 2013-02-25 18:04:21 -08:00
Matei Zaharia c45d6f1248 Merge pull request #501 from tdas/master 
Fixed bug in BlockManager and added a testcase
 2013-02-25 17:35:31 -08:00