ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3337 commits 6 branches 181 tags 397 MiB
Commit graph

3337 commits

This branch
This branch
All branches
Author SHA1 Message Date
Patrick Wendell c537e869f3 Missing logo file 2013-06-27 22:02:03 -07:00
Patrick Wendell c767e74370 Removing incorrect test statement 2013-06-27 21:48:58 -07:00
Patrick Wendell 62c2c6b856 Forcing Jetty to run as daemon 2013-06-27 21:47:22 -07:00
Patrick Wendell a55190d314 Adding better tabs for UI headers. 2013-06-27 19:14:51 -07:00
Patrick Wendell 362d996c81 Handful of changes based on matei's review 
- Avoid exception when no tasks have finished for a stage
- Adding DOCTYPE so css renders properly
- Adding progress slider
 2013-06-27 19:14:28 -07:00
Patrick Wendell 92a4c2a5f6 Fixing bug in local scheduler time recording 2013-06-27 12:33:06 -07:00
Matei Zaharia aea727f68d Simplify Python docs a little to do substring search 2013-06-26 21:15:09 -07:00
Matei Zaharia 03906f7f0a Fixes to compute-classpath on Windows 2013-06-26 17:40:22 -07:00
Matei Zaharia e49bc8ca8c Merge pull request #663 from stephenh/option_and_getenv 
Be cute with Option and getenv.
 2013-06-26 11:13:33 -07:00
Stephen Haberman d7011632d1 Wrap lines. 2013-06-26 12:35:57 -05:00
Patrick Wendell ee692482a6 One more private class 2013-06-26 09:07:32 -07:00
Patrick Wendell a59c15a37e Adding config option for retained stages 2013-06-26 08:54:57 -07:00
Patrick Wendell 274193664a Bumping timeouts 2013-06-26 08:51:28 -07:00
Patrick Wendell b14ad509ba Moving static ui package 2013-06-26 08:46:51 -07:00
Patrick Wendell 2cbaa0734b Making all new classes package private 2013-06-26 08:44:55 -07:00
Stephen Haberman d11025dc6a Be cute with Option and getenv. 2013-06-26 09:53:35 -05:00
Matei Zaharia 32370da4e4 Don't use forward slash in exclusion for JAR signature files 2013-06-25 22:08:19 -04:00
Matei Zaharia 9f0d913295 Refactored tests to share SparkContexts in some of them 
Creating these seems to take a while and clutters the output with Akka
stuff, so it would be nice to share them.
 2013-06-25 19:18:30 -04:00
Matei Zaharia 2bd04c3513 Formatting 2013-06-25 18:37:14 -04:00
Matei Zaharia f2263350ed Added a local-cluster mode test to ReplSuite 2013-06-25 18:35:35 -04:00
Matei Zaharia 6c8d1b2ca6 Fix computation of classpath when we launch java directly 
The previous version assumed that a CLASSPATH environment variable was
set by the "run" script when launching the process that starts the
ExecutorRunner, but unfortunately this is not true in tests. Instead, we
factor the classpath calculation into an extenral script and call that.

NOTE: This includes a Windows version but hasn't yet been tested there.
 2013-06-25 18:21:00 -04:00
James Phillpotts 176193b1e8 Fix usage and parameter extraction 2013-06-25 23:06:15 +01:00
Evan Chan 8dd78f80b5 Manually merge spark/master commit d92d3f7938 2013-06-25 15:01:06 -07:00
James Phillpotts 366572edca Include a default OAuth implementation, and update examples and JavaStreamingContext 2013-06-25 22:59:34 +01:00
Evan Chan d2f46ac680 Merge branch 'master' into 2013-06/assembly-jar-deploy 
Conflicts:
	run
 2013-06-25 14:50:16 -07:00
Matei Zaharia 15b00914c5 Some fixes to the launch-java-directly change: 
- Split SPARK_JAVA_OPTS into multiple command-line arguments if it
  contains spaces; this splitting follows quoting rules in bash
- Add the Scala JARs to the classpath if they're not in the CLASSPATH
  variable because the ExecutorRunner is launched with "scala" (this can
  happen when using local-cluster URLs in spark-shell)
 2013-06-25 17:17:27 -04:00
Matei Zaharia 7680ce0bd6 Fixed deprecated use of expect in SizeEstimatorSuite 2013-06-25 16:11:44 -04:00
Matei Zaharia 7e0191c6ea Merge remote-tracking branch 'cgrothaus/SPARK-698' 
Conflicts:
	run
 2013-06-25 15:47:40 -04:00
Evan Chan 982a686ff8 Add -m and -c <cores> options to spark-shell for convenience 2013-06-25 11:14:21 -07:00
Matei Zaharia f5e32ed13a Merge pull request #661 from mesos/streaming 
Kafka fixes and DStream.count fix for master
 2013-06-25 09:16:57 -07:00
Evan Chan 243d71cb06 Add deploy/testing procedure 2013-06-25 00:15:58 -07:00
Evan Chan c3d11d0d57 Get rid of debugging statements 2013-06-25 00:02:00 -07:00
Tathagata Das c89af0a7f9 Merge branch 'master' into streaming 
Conflicts:
	.gitignore
 2013-06-24 23:57:47 -07:00
Evan Chan cee05a1748 Copy restore-TTY functions from Scala script so binary distros don't need 'scala' installed 2013-06-24 23:55:09 -07:00
Tathagata Das 48c7e373c6 Minor formatting fixes 2013-06-24 23:11:04 -07:00
Tathagata Das 1249e9153b Merge pull request #572 from Reinvigorate/sm-block-interval 
Adding spark.streaming.blockInterval property
 2013-06-24 21:46:33 -07:00
Tathagata Das cfcda95f86 Merge pull request #571 from Reinvigorate/sm-kafka-serializers 
Surfacing decoders on KafkaInputDStream
 2013-06-24 21:44:50 -07:00
Tathagata Das 575aff6b71 Merge pull request #567 from Reinvigorate/sm-count-fix 
Fixing count() in Spark Streaming
 2013-06-24 21:35:50 -07:00
Patrick Wendell d66bd6f885 Adding another unit test to Web UI suite 2013-06-24 17:12:55 -07:00
Evan Chan 81df20e5b4 Script to create binary distribution for Spark 2013-06-24 17:05:37 -07:00
Patrick Wendell f7389330c3 Allowing for requested port on construction 2013-06-24 16:51:52 -07:00
Patrick Wendell 42157027f2 A few bug fixes and a unit test 2013-06-24 16:25:05 -07:00
Evan Chan 0bcaf03605 Split out source distro CLASSPATH logic to a separate script 2013-06-24 15:39:52 -07:00
Evan Chan 4cda8f865a Add simple usage to start-slave script 2013-06-24 15:14:48 -07:00
Patrick Wendell a4248138b4 Minor style cleanup 2013-06-24 14:22:28 -07:00
Patrick Wendell b5e6e8bcc8 Cleaning up some code for Job Progress 2013-06-24 14:13:24 -07:00
Patrick Wendell 93e8ed85aa Work around for initalization issue 2013-06-24 13:11:18 -07:00
Patrick Wendell f6e64b5cd6 Updating based on changes to JobLogger (and one small change to JobLogger) 2013-06-24 12:40:41 -07:00
James Phillpotts 8955787a59 Twitter API v1 is retired - username/password auth no longer possible 2013-06-24 09:15:17 +01:00
Matei Zaharia 78ffe164b3 Clone the zero value for each key in foldByKey 
The old version reused the object within each task, leading to
overwriting of the object when a mutable type is used, which is expected
to be common in fold.

Conflicts:

	core/src/test/scala/spark/ShuffleSuite.scala
 2013-06-23 10:26:53 -07:00