Matei Zaharia
1069740264
Added a jarOfObject method to get the JAR of the class that an object
...
belongs to, which seems like a more common case.
2011-08-29 23:27:10 -07:00
Matei Zaharia
0c965a102e
Removed a debugging line
2011-08-29 23:13:33 -07:00
Matei Zaharia
0aa23bf17e
Added a convenience method for getting the JAR file that loaded a class
...
(useful for jobs to pass their own JAR files to SparkContext).
2011-08-29 22:59:44 -07:00
Matei Zaharia
9b7215d74a
Only include JAR files in lib in the unmanaged classpath.
2011-08-29 22:59:18 -07:00
Matei Zaharia
4cc5e51e28
Use the new SBT 0.10 assembly plugin instead of the DepJar code we had.
2011-08-29 16:13:26 -07:00
Matei Zaharia
a161f00610
Made a log message slightly less ugly
2011-08-27 16:58:54 -07:00
Matei Zaharia
3759bcd061
New mesos.jar
2011-08-10 14:03:48 -07:00
Matei Zaharia
c22043f150
Minor fix: can use >= when checking memory
2011-08-02 19:11:17 -07:00
Matei Zaharia
f6c8c2a394
Merge pull request #77 from ijuma/issue69
...
Enable -optimize in the build
2011-08-02 15:47:58 -07:00
Matei Zaharia
cfc6000ede
Merge pull request #76 from ijuma/deprecations-scala-2.9
...
Fix code not to use code deprecated in Scala 2.9
2011-08-02 15:47:42 -07:00
Ismael Juma
2b6fd3198d
Fix issue #69 : Enable -optimize in the build
2011-08-02 10:34:41 +01:00
Ismael Juma
59c7131dff
Add note about why we can't enable -deprecation switch.
2011-08-02 10:26:17 +01:00
Ismael Juma
065043a14f
Use process instead of main as the latter is deprecated.
2011-08-02 10:26:03 +01:00
Ismael Juma
6ff57f5594
Use scala.math instead of Math as the latter is deprecated.
2011-08-02 10:25:47 +01:00
Ismael Juma
620de2dd1d
Change currentThread to Thread.currentThread as the former is deprecated.
2011-08-02 10:25:16 +01:00
Ismael Juma
0fba22b3d2
Fix issue #65 : Change @serializable to extends Serializable in 2.9 branch
...
Note that we use scala.Serializable introduced in Scala 2.9 instead of
java.io.Serializable. Also, case classes inherit from scala.Serializable by
default.
2011-08-02 10:16:33 +01:00
Matei Zaharia
2e57338896
Merge branch 'scala-2.9'
2011-08-01 15:27:08 -07:00
Matei Zaharia
a0b7865037
Merge pull request #75 from ijuma/scala-2.9
...
Workaround for scalac bug and publishTo configuration
2011-08-01 15:26:31 -07:00
Matei Zaharia
711575391d
Merge branch 'scala-2.9'
...
Conflicts:
project/build/SparkProject.scala
2011-08-01 15:25:26 -07:00
Matei Zaharia
4050d661c5
Updated to newest Mesos API, which includes better memory accounting
...
by specifying per-executor memory.
2011-08-01 13:54:48 -07:00
Ismael Juma
3f5d7b5d11
Add publishTo configuration that publishes to target directory.
...
Until Spark is available in a Maven repository, this makes it easy to deploy to
e.g. GitHub pages by copying target/maven to the repository.
2011-07-31 20:17:58 +01:00
Ismael Juma
7565cc2e33
Add type annotation to result of depJarSettings to workaround scalac bug.
2011-07-31 20:17:58 +01:00
Matei Zaharia
d12122502b
Various improvements to Kryo serializer:
...
- Replaced modified Kryo version with the standard one augmented with
the kryo-serializers package, which includes support for classes with
no-arg constructors (that was why we had a modified Kryo before)
- The kryo-serializers version also fixes issue #72 .
- Added a bunch of tests.
- Serialize maps and a few other common types properly by default.
2011-07-21 22:09:33 -07:00
Matei Zaharia
baa72e2747
Removed a debug statement that slipped in as a println
2011-07-21 16:09:33 -07:00
Matei Zaharia
2bfd7931e8
Merge branch 'new-rdds-protobuf'
...
Conflicts:
core/src/main/scala/spark/Executor.scala
core/src/main/scala/spark/RDD.scala
2011-07-21 16:08:39 -07:00
Ismael Juma
ecb8b69fa0
sbt 0.10.1 calls update automatically so remove it from the documentation.
2011-07-21 01:04:55 +01:00
Ismael Juma
fc0a2c8db8
Add and configure junit_xml_listener as a replacement for XmlTestReport.
2011-07-21 01:04:29 +01:00
Ismael Juma
51673ca62e
Introduce DepJarPlugin based on AssemblyPlugin and use it in SparkBuild.
2011-07-18 10:34:51 +01:00
Ismael Juma
c71fc27c74
Update sbt-idea to non-snapshot version and uncomment sbt-assembly dependency.
2011-07-18 00:17:21 +01:00
Ismael Juma
8531c2a079
Update test dependencies.
2011-07-18 00:16:08 +01:00
Ismael Juma
635f501492
Fix copy & paste error in version.
2011-07-18 00:13:37 +01:00
Ismael Juma
f686e3dacb
Initial work on converting build to SBT 0.10.1
2011-07-15 03:38:30 +01:00
Matei Zaharia
cf8f5de61b
Merge branch 'master' into scala-2.9
...
Conflicts:
project/build.properties
repl/src/main/scala/spark/repl/SparkInterpreterLoop.scala
2011-07-14 17:48:56 -04:00
Matei Zaharia
02678724a4
Update version number to 0.4-SNAPSHOT
2011-07-14 17:47:39 -04:00
Matei Zaharia
7c77b2fa6a
Merge branch 'master' into scala-2.9
...
Conflicts:
project/build.properties
2011-07-14 17:39:34 -04:00
Matei Zaharia
c86af80022
Change version to 0.3
2011-07-14 17:38:43 -04:00
Matei Zaharia
1450fd74d9
Merge branch 'master' into scala-2.9
2011-07-14 17:37:24 -04:00
Matei Zaharia
ccf48388cd
Lowered default number of splits for files
2011-07-14 17:37:04 -04:00
Matei Zaharia
146a18c2a4
Merge branch 'master' into scala-2.9
2011-07-14 17:29:17 -04:00
Matei Zaharia
c8eb8b2b90
Set class loader for remote actors to fix a bug that happens in 2.9
2011-07-14 17:29:11 -04:00
Matei Zaharia
8ea67307b9
Merge branch 'master' into scala-2.9
2011-07-14 14:47:12 -04:00
Matei Zaharia
e4c3402d2d
Renamed ParallelArray to ParallelCollection
2011-07-14 14:47:01 -04:00
Matei Zaharia
9ac461d85d
Remove RDD.toString because it looked confusing
2011-07-14 14:39:32 -04:00
Matei Zaharia
797b4547c3
Fix tracking of updates in accumulators to solve an issue that would manifest in the 2.9 interpreter
2011-07-14 14:08:34 -04:00
Matei Zaharia
3efd9e94d8
Merge branch 'master' into scala-2.9
2011-07-14 12:42:57 -04:00
Matei Zaharia
0ccfe20755
Forgot to add a file
2011-07-14 12:42:50 -04:00
Matei Zaharia
38f38dda5b
Merge branch 'master' into scala-2.9
2011-07-14 12:42:02 -04:00
Matei Zaharia
969644df8e
Cleaned up a few issues to do with default parallelism levels. Also
...
renamed HadoopFileWriter to HadoopWriter (since it's not only for files)
and fixed a bug for lookup().
2011-07-14 12:40:56 -04:00
Matei Zaharia
2fb906e8e5
Merge branch 'master' into scala-2.9
2011-07-14 00:20:14 -04:00
Matei Zaharia
2604939f64
Simplified and documented code a little and added test
2011-07-14 00:19:00 -04:00
