Matei Zaharia
|
112655f032
|
Merge pull request #121 from rxin/kryo-closure
Added an option (spark.closure.serializer) to specify the serializer for closures.
|
2012-04-10 14:21:02 -07:00 |
|
Reynold Xin
|
d295ccb43c
|
Added a closureSerializer field in SparkEnv and use it to serialize
tasks.
|
2012-04-10 13:29:46 -07:00 |
|
Reynold Xin
|
968f75f6af
|
Added an option (spark.closure.serializer) to specify the serializer for
closures. This enables using Kryo as the closure serializer.
|
2012-04-09 21:59:56 -07:00 |
|
Matei Zaharia
|
a69c0738d1
|
Merge branch 'master' into mesos-0.9
|
2012-04-08 23:41:36 -07:00 |
|
Matei Zaharia
|
a633974143
|
Merge branch 'master' of github.com:mesos/spark
|
2012-04-08 23:41:25 -07:00 |
|
Matei Zaharia
|
0229d5390f
|
Merge branch 'master' into mesos-0.9
|
2012-04-08 23:39:37 -07:00 |
|
Matei Zaharia
|
d401e1b3e8
|
Fix a possible deadlock in MesosScheduler
|
2012-04-08 23:38:49 -07:00 |
|
Matei Zaharia
|
a7d6ffc5fd
|
Merge pull request #119 from mesos/report-cache-events
Report entry dropping in BoundedMemoryCache
|
2012-04-06 15:59:29 -07:00 |
|
Ankur Dave
|
7be1c7b331
|
Report entry dropping in BoundedMemoryCache
|
2012-04-06 15:49:32 -07:00 |
|
Matei Zaharia
|
a8bb324ed9
|
Merge branch 'master' into mesos-0.9
|
2012-04-05 14:53:22 -07:00 |
|
Matei Zaharia
|
816d4e5840
|
Pass local IP address instead of hostname in spark.master.host. Fixes #117.
|
2012-04-05 14:53:17 -07:00 |
|
Matei Zaharia
|
335a6036ad
|
Converted some tabs to spaces
|
2012-04-05 11:58:01 -07:00 |
|
Matei Zaharia
|
acaf99cb3a
|
Merge branch 'master' into mesos-0.9
|
2012-03-30 13:39:47 -04:00 |
|
Matei Zaharia
|
8c95a85438
|
Use Runtime.maxMemory instead of Runtime.totalMemory in
BoundedMemoryCache, in case the JVM was not started with its initial
heap size equaling its maximum one (-Xms == -Xmx).
|
2012-03-30 13:39:35 -04:00 |
|
Matei Zaharia
|
03d5b3b48d
|
Use Runtime.maxMemory instead of Runtime.totalMemory in
BoundedMemoryCache, in case the JVM was not started with its initial
heap size equaling its maximum one (-Xms == -Xmx).
|
2012-03-30 13:38:19 -04:00 |
|
Matei Zaharia
|
95fb1a16b8
|
Use Mesos 0.9 RC3 JAR and protobuf 2.4.1
|
2012-03-30 11:38:49 -04:00 |
|
Matei Zaharia
|
dfa3b6b544
|
Fixes to work with the very latest Mesos 0.9 API
|
2012-03-29 22:12:35 -04:00 |
|
Matei Zaharia
|
4d52cc6738
|
Merge branch 'master' into mesos-0.9
|
2012-03-29 21:29:39 -04:00 |
|
Reynold Xin
|
d46f662c5b
|
Merge branch 'master' of github.com:mesos/spark
|
2012-03-29 15:22:17 -07:00 |
|
Reynold Xin
|
42dcdbcb2f
|
Removed the extra spaces in OrderedRDDFunctions and SortedRDD.
|
2012-03-29 15:21:57 -07:00 |
|
Matei Zaharia
|
ca5c19c1ba
|
Remove dependency on Akka
|
2012-03-29 01:03:34 -04:00 |
|
Reynold Xin
|
90418b70ff
|
Added sbt-assembly for spark-repl project so we can generate an
assembled jar for Shark.
|
2012-03-22 18:46:31 -07:00 |
|
Matei Zaharia
|
ca64a7ae03
|
Documentation
|
2012-03-17 13:51:29 -07:00 |
|
Matei Zaharia
|
36c7db72bc
|
Documentation
|
2012-03-17 13:49:55 -07:00 |
|
Matei Zaharia
|
08cda89e8a
|
Further fixes to how Mesos is found and used
|
2012-03-17 13:39:14 -07:00 |
|
Matei Zaharia
|
3c3fdf6eca
|
Merge branch 'master' into mesos-0.9
|
2012-03-17 13:09:21 -07:00 |
|
Matei Zaharia
|
c7af538ac1
|
Some fixes to sorting for when the RDD has fewer elements than the
number of partitions we ask to partition it into. Also, removed a test
that was taking way too long to run.
|
2012-03-17 13:08:36 -07:00 |
|
Matei Zaharia
|
a099a63a8a
|
Initial work to make Spark compile with Mesos 0.9 and Hadoop 1.0
|
2012-03-17 12:31:34 -07:00 |
|
Matei Zaharia
|
a5e2b6a6bd
|
Merge pull request #112 from cengle/master
Changed HadoopRDD to get key and value containers from the RecordReader instead of through reflection
|
2012-03-06 13:38:32 -08:00 |
|
Matei Zaharia
|
97eee50825
|
Fixes a nasty bug that could happen when tasks fail, because calling
wait() with a timeout of 0 on a Java object means "wait forever".
|
2012-03-01 13:43:17 -08:00 |
|
Cliff Engle
|
dd68cb6099
|
Get key and value container from RecordReader
|
2012-02-29 16:33:23 -08:00 |
|
Matei Zaharia
|
1e10df0a46
|
Merge pull request #111 from alupher/master
Adding sorting to RDDs
|
2012-02-24 15:50:14 -08:00 |
|
Antonio
|
0d93d95bcf
|
Removed unnecessary import
|
2012-02-21 19:57:12 -08:00 |
|
Antonio
|
2990298f71
|
Added sorting testing suite
|
2012-02-21 19:54:21 -08:00 |
|
Matei Zaharia
|
aa04f87cd2
|
Added support for parallel execution of jobs in DAGScheduler.
|
2012-02-19 22:50:23 -08:00 |
|
Antonio
|
620798161b
|
Added fixes to sorting
|
2012-02-13 00:07:39 -08:00 |
|
Matei Zaharia
|
2587ce1690
|
Fixed a deadlock that occured with MesosScheduler due to an earlier
synchronization change
|
2012-02-11 21:22:45 -08:00 |
|
Antonio
|
e93f622665
|
Added sorting by key for pair RDDs
|
2012-02-11 00:56:28 -08:00 |
|
Matei Zaharia
|
98f008b721
|
Formatting fixes
|
2012-02-10 10:52:03 -08:00 |
|
Matei Zaharia
|
7660a8b12f
|
Merge branch 'formatting'
Conflicts:
core/src/main/scala/spark/DAGScheduler.scala
core/src/main/scala/spark/SimpleShuffleFetcher.scala
core/src/main/scala/spark/SparkContext.scala
|
2012-02-10 10:42:14 -08:00 |
|
haoyuan
|
194c42ab79
|
Code format.
|
2012-02-10 08:19:53 -08:00 |
|
Matei Zaharia
|
8f5ed51234
|
Delete Spark's temporary directories when the JVM exits.
|
2012-02-09 22:58:24 -08:00 |
|
Matei Zaharia
|
c0a0df3285
|
Made the default cache BoundedMemoryCache, and reduced its default size
|
2012-02-09 22:32:02 -08:00 |
|
Matei Zaharia
|
a766780f4c
|
Added some tests for multithreaded access to Spark.
|
2012-02-09 22:27:53 -08:00 |
|
Matei Zaharia
|
0e93891d3d
|
Replaced LocalFileShuffle with a non-singleton ShuffleManager class
and made DAGScheduler automatically set SparkEnv.
|
2012-02-09 22:14:56 -08:00 |
|
haoyuan
|
445e0bb1b5
|
Format the code a bit mroe.
|
2012-02-09 15:50:26 -08:00 |
|
haoyuan
|
651932e703
|
Format the code as coding style agreed by Matei/TD/Haoyuan
|
2012-02-09 13:26:23 -08:00 |
|
Matei Zaharia
|
e02dc83a5b
|
IO optimizations
|
2012-02-06 20:40:39 -08:00 |
|
Matei Zaharia
|
c40e766368
|
Use java.util.HashMap in shuffles
|
2012-02-06 19:20:25 -08:00 |
|
Matei Zaharia
|
d6ec664b48
|
Add dependency on fastutil and update Guava
|
2012-02-06 15:37:27 -08:00 |
|