ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
25030 commits 6 branches 181 tags 397 MiB
Commit graph

25030 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia b49d1be65b Ensure logging is initialized before any Spark threads run in the REPL 2011-05-31 23:54:48 -07:00
Matei Zaharia 10fe324845 Merge remote-tracking branch 'origin/master' into scala-2.9 2011-05-31 23:48:11 -07:00
Matei Zaharia 5166d76843 Ensure logging is initialized before spawning any threads to fix issue #45 2011-05-31 23:47:32 -07:00
Matei Zaharia 96daa31a01 Pass quoted arguments properly to run 2011-05-31 23:34:16 -07:00
Matei Zaharia 7862995569 Give SBT a bit more memory so it can do a update / compile / test in one JVM 2011-05-31 23:33:47 -07:00
Matei Zaharia 90f924202b Another fix ported forward for the REPL 2011-05-31 23:11:49 -07:00
Matei Zaharia 3854d23dd4 Pass quoted arguments properly to run 2011-05-31 22:17:48 -07:00
Matei Zaharia 8012e388a8 Give SBT a bit more memory so it can do a update / compile / test in one JVM 2011-05-31 22:17:33 -07:00
Ismael Juma 3def9fdb96 Upgrade to scalacheck 1.9. 2011-05-31 22:11:33 -07:00
Matei Zaharia 0afd35a8dd Some docs in ClosureCleaner 2011-05-31 22:06:30 -07:00
Matei Zaharia 73975d7491 Further fixes to interpreter (adding in some code generation changes I 
missed before and setting SparkEnv properly on the threads that execute
each line in the 2.9 interpreter).
 2011-05-31 22:05:24 -07:00
Matei Zaharia d52660c969 Ported code generation changes from 2.8 interpreter (to use a class for 
each line's object rather than a singleton object so that we can ship
these classes to worker nodes). This is pretty hairy stuff, which would
be nice to avoid in the future by integrating with the interpreter some
other way.
 2011-05-31 19:23:15 -07:00
Matei Zaharia beb9c117f0 Merge branch 'master' into scala-2.9 
Conflicts:
	project/build/SparkProject.scala
 2011-05-31 19:23:07 -07:00
Matei Zaharia bcce6e8d01 Various work to use the 2.9 interpreter 2011-05-31 17:31:51 -07:00
Matei Zaharia 8b0390d344 Instantiate NullWritable properly in HadoopFile 2011-05-30 23:54:14 -07:00
Matei Zaharia 4096c2287e Various fixes 2011-05-29 18:46:01 -07:00
Matei Zaharia ef706ae959 Merge branch 'master' into new-rdds-protobuf 
Conflicts:
	run
 2011-05-29 16:20:23 -07:00
Matei Zaharia c501cff924 Executor was looking for the wrong constructor for ExecutorClassLoader 2011-05-29 16:15:59 -07:00
Matei Zaharia 50ac1d2a40 Merge remote-tracking branch 'ijuma/issue51' 2011-05-29 15:41:12 -07:00
Ismael Juma 0c62ee4321 Depend on jetty-server in compile scope and upgrade to 7.4.2. 
As Matei described: "We're using Jetty to run an HTTP server, not to embed Spark
in a webapp"
 2011-05-29 20:12:50 +01:00
Matei Zaharia 22c8b84d8b Merge pull request #53 from ijuma/master 
Use explicit asInstanceOf instead of misleading unchecked pattern matching.
 2011-05-27 10:20:54 -07:00
Ismael Juma 2c691295bf SCALA_VERSION in run should be 2.9.0-1 2011-05-27 14:59:23 +01:00
Ismael Juma 1d75c6060a Update to Scala 2.9.0-1 and disable repl module for now. 
The repl module requires more complex work.
 2011-05-27 14:59:23 +01:00
Ismael Juma e3b323321d Use ManagedStyle.Maven. 2011-05-27 14:56:01 +01:00
Ismael Juma 3a6b0b8a57 Publish javadoc and sources. 2011-05-27 14:55:51 +01:00
Ismael Juma 59f1f42a9a Update run to work with SBT managed dependencies and the newly introduced repl module. 2011-05-27 11:22:59 +01:00
Ismael Juma 3af6003c87 Update sbt to 0.7.7. 2011-05-27 11:22:59 +01:00
Ismael Juma 1396678baa Move REPL classes to separate module. 2011-05-27 11:22:50 +01:00
Ismael Juma 3e8114ddbd Change project.organization to org.spark-project to fit Maven convention. 2011-05-27 11:22:10 +01:00
Ismael Juma 7b7dfdb085 Set project.version to 0.3-SNAPSHOT. 2011-05-27 11:22:10 +01:00
Ismael Juma 051da8b4ad Delete liblzf from lib as it's no longer used. 2011-05-27 11:22:10 +01:00
Ismael Juma ae1a1f91f1 Remove several dependencies from git and configure them as SBT managed dependencies. 
Upgrade some of the dependencies while at it.
 2011-05-27 11:22:01 +01:00
Ismael Juma 164ef4c751 Use explicit asInstanceOf instead of misleading unchecked pattern matching. 
Also enable -unchecked warnings in SBT build file.
 2011-05-27 07:57:10 +01:00
Matei Zaharia cfbe2da1a6 Merge pull request #52 from ijuma/master 
Fix deprecations when compiled with Scala 2.8.1
 2011-05-26 23:53:10 -07:00
Ismael Juma 89c8ea2bb2 Replace deprecated - and -- with suggested filterNot (which is uglier). 2011-05-26 22:22:37 +01:00
Ismael Juma 94f05683bd Replace deprecated first with head. 2011-05-26 22:13:41 +01:00
Ismael Juma 26000af4fa Replace deprecated fromFunction with either tabulate or fill. 
tabulate used if indexed used by function and fill otherwise.
 2011-05-26 22:12:11 +01:00
Ismael Juma 0b6a862b68 Use math instead of Math as the latter is deprecated. 2011-05-26 22:06:36 +01:00
Ismael Juma 1f27d94c48 Use Array.iterator instead of Iterator.fromArray as the latter is deprecated. 2011-05-26 22:04:42 +01:00
Ismael Juma 1993a8e556 Use += instead of + for mutable sequences as the latter is deprecated. 2011-05-26 21:59:48 +01:00
Ismael Juma 222729171e Upgrade sbt-idea to 0.4.0. 2011-05-26 21:48:08 +01:00
Matei Zaharia f707856522 Removed java-opts.template 2011-05-24 15:59:01 -07:00
root 5ef938615f Initial work on making stuff compile with protobuf Mesos 2011-05-24 22:27:08 +00:00
Matei Zaharia cec427e777 Fixed a bug with preferred locations having changed meaning in new RDDs 2011-05-22 17:12:29 -07:00
Matei Zaharia 4c888b2933 Fix queue type for executor 2011-05-22 16:42:05 -07:00
Matei Zaharia bea3a33012 doc tweak 2011-05-22 16:03:41 -07:00
Matei Zaharia 9bde5a54cb class loader fix 2011-05-22 16:00:41 -07:00
Matei Zaharia 91c07a33d9 Various fixes to serialization 2011-05-21 22:50:08 -07:00
Matei Zaharia f61b61c4ac Merge branch 'master' into new-rdds 2011-05-21 21:25:58 -07:00
Matei Zaharia 24a1e7f838 Scheduler can now recover from lost map outputs 2011-05-20 00:19:53 -07:00