Josh Rosen
|
2c966c98fb
|
Change numSplits to numPartitions in PySpark.
|
2013-02-24 13:25:09 -08:00 |
|
Matei Zaharia
|
3b9f929467
|
Merge pull request #468 from haitaoyao/master
support customized java options for Master, Worker, Executor, and Repl
|
2013-02-23 23:38:15 -08:00 |
|
Matei Zaharia
|
beb79dbbfd
|
Merge pull request #473 from stephenh/subtract
Add RDD.subtract.
|
2013-02-23 23:25:27 -08:00 |
|
Stephen Haberman
|
37c7a71f9c
|
Add subtract to JavaRDD, JavaDoubleRDD, and JavaPairRDD.
|
2013-02-24 00:27:53 -06:00 |
|
Stephen Haberman
|
f442e7d83c
|
Update for split->partition rename.
|
2013-02-24 00:27:14 -06:00 |
|
Stephen Haberman
|
cec87a0653
|
Merge branch 'master' into subtract
|
2013-02-23 23:27:55 -06:00 |
|
Matei Zaharia
|
8a0ca6f05c
|
Merge pull request #491 from pwendell/java-vector
Adding Java-accessible methods to Vector.scala
|
2013-02-23 15:55:06 -08:00 |
|
Patrick Wendell
|
931f439be9
|
Responding to code review
|
2013-02-23 15:40:41 -08:00 |
|
Patrick Wendell
|
f51b0f93f2
|
Adding Java-accessible methods to Vector.scala
This is needed for the Strata machine learning tutorial (and
also is generally helpful).
|
2013-02-23 13:26:59 -08:00 |
|
Matei Zaharia
|
d942d39072
|
Handle exceptions in RecordReader.close() better (suggested by Jim
Donahue)
|
2013-02-23 11:19:07 -08:00 |
|
Matei Zaharia
|
c89824046a
|
Merge pull request #490 from woggling/conn-death
Detect when SendingConnections disconnect even if we aren't sending to them
|
2013-02-22 22:58:19 -08:00 |
|
Charles Reiss
|
50cf8c8b79
|
Add fault tolerance test that uses replicated RDDs.
|
2013-02-22 16:11:53 -08:00 |
|
Charles Reiss
|
c8a7886921
|
Detect when SendingConnections drop by trying to read them.
Comment fix
|
2013-02-22 16:11:52 -08:00 |
|
Matei Zaharia
|
d4d7993bf5
|
Several fixes to the work to log when no resources can be used by a job.
Fixed some of the messages as well as code style.
|
2013-02-22 15:51:37 -08:00 |
|
Matei Zaharia
|
f33662c133
|
Merge remote-tracking branch 'pwendell/starvation-check'
Also fixed a bug where master was offering executors on dead workers
Conflicts:
core/src/main/scala/spark/deploy/master/Master.scala
|
2013-02-22 15:27:41 -08:00 |
|
Matei Zaharia
|
7341de0d48
|
Merge pull request #475 from JoshRosen/spark-668
Remove hack workaround for SPARK-668
|
2013-02-22 14:56:18 -08:00 |
|
Matei Zaharia
|
fcf7fa8448
|
Merge pull request #489 from pwendell/SPARK-702
SPARK-702: Replace Function --> JFunction in JavaAPI Suite.
|
2013-02-22 14:55:46 -08:00 |
|
Patrick Wendell
|
f8c3a03d55
|
SPARK-702: Replace Function --> JFunction in JavaAPI Suite.
In a few places the Scala (rather than Java) function class is used.
|
2013-02-22 12:54:15 -08:00 |
|
haitao.yao
|
8215b95547
|
Merge branch 'mesos'
|
2013-02-21 10:07:24 +08:00 |
|
Matei Zaharia
|
05bc02e80b
|
Merge pull request #482 from woggling/shutdown-exceptions
Don't call System.exit over uncaught exceptions from shutdown hooks
|
2013-02-19 20:56:15 -08:00 |
|
haitao.yao
|
6a3d44c673
|
Merge branch 'mesos'
|
2013-02-20 10:23:58 +08:00 |
|
Charles Reiss
|
092c631fa8
|
Pull detection of being in a shutdown hook into utility function.
|
2013-02-19 17:49:55 -08:00 |
|
Matei Zaharia
|
8a992226bd
|
Merge pull request #484 from andyk/master
Fixes a broken link in documentation to issue tracker
|
2013-02-19 17:07:24 -08:00 |
|
Matei Zaharia
|
a3e86b2b1f
|
Merge pull request #483 from rxin/splitpruningrdd2
Added a method to create PartitionPruningRDD.
|
2013-02-19 17:07:00 -08:00 |
|
Andy Konwinski
|
ecd137a72d
|
Fixes link to issue tracker in documentation page "Contributing to Spark".
|
2013-02-19 16:58:02 -08:00 |
|
Reynold Xin
|
130f704baf
|
Added a method to create PartitionPruningRDD.
|
2013-02-19 16:03:52 -08:00 |
|
Charles Reiss
|
d0588bd6d7
|
Catch/log errors deleting temp dirs
|
2013-02-19 13:04:06 -08:00 |
|
Charles Reiss
|
687581c3ec
|
Paranoid uncaught exception handling for exceptions during shutdown
|
2013-02-19 13:03:02 -08:00 |
|
Matei Zaharia
|
03d847999e
|
Merge pull request #477 from shivaram/ganglia-port-change
Ganglia port change
|
2013-02-18 20:25:48 -08:00 |
|
haitao.yao
|
7c129388fb
|
Merge branch 'mesos'
|
2013-02-19 11:22:24 +08:00 |
|
Shivaram Venkataraman
|
6cba5a48b0
|
Print cluster url after setup completes
|
2013-02-18 18:30:36 -08:00 |
|
Shivaram Venkataraman
|
e7cdf7a6a4
|
Print ganglia url after setup
|
2013-02-18 17:15:22 -08:00 |
|
Shivaram Venkataraman
|
03f45a18d5
|
Use port 5080 for httpd/ganglia
|
2013-02-18 16:56:01 -08:00 |
|
Matei Zaharia
|
7151e1e4c8
|
Rename "jobs" to "applications" in the standalone cluster
|
2013-02-17 23:23:08 -08:00 |
|
Matei Zaharia
|
06e5e6627f
|
Renamed "splits" to "partitions"
|
2013-02-17 22:13:26 -08:00 |
|
Matei Zaharia
|
455d015076
|
Clean up EC2 script options a bit
|
2013-02-17 16:53:12 -08:00 |
|
Matei Zaharia
|
08e444df0e
|
Change EC2 script to use 0.6 AMIs by default, for now
|
2013-02-17 14:01:48 -08:00 |
|
Matei Zaharia
|
2a907dceb3
|
Merge pull request #421 from shivaram/spark-ec2-change
Switch spark_ec2.py to use the new spark-ec2 scripts.
|
2013-02-17 13:48:43 -08:00 |
|
Matei Zaharia
|
340cc54e47
|
Merge pull request #471 from stephenh/parallelrdd
Move ParallelCollection into spark.rdd package.
|
2013-02-16 16:39:15 -08:00 |
|
Matei Zaharia
|
3260b6120e
|
Merge pull request #470 from stephenh/morek
Make CoGroupedRDDs explicitly have the same key type.
|
2013-02-16 16:38:38 -08:00 |
|
Stephen Haberman
|
924f47dd11
|
Add RDD.subtract.
Instead of reusing the cogroup primitive, this adds a SubtractedRDD
that knows it only needs to keep rdd1's values (per split) in memory.
|
2013-02-16 13:38:42 -06:00 |
|
Stephen Haberman
|
e7713adb99
|
Move ParallelCollection into spark.rdd package.
|
2013-02-16 13:20:48 -06:00 |
|
Stephen Haberman
|
ae2234687d
|
Make CoGroupedRDDs explicitly have the same key type.
|
2013-02-16 13:10:31 -06:00 |
|
Matei Zaharia
|
9d979fb630
|
Merge pull request #469 from stephenh/samepartitionercombine
If combineByKey is using the same partitioner, skip the shuffle.
|
2013-02-16 10:07:42 -08:00 |
|
Stephen Haberman
|
4328873294
|
Add assertion about dependencies.
|
2013-02-16 01:16:40 -06:00 |
|
Stephen Haberman
|
c34b8ad2c5
|
Avoid a shuffle if combineByKey is passed the same partitioner.
|
2013-02-16 00:54:03 -06:00 |
|
haitao.yao
|
858784459f
|
support customized java options for master, worker, executor, repl shell
|
2013-02-16 14:42:06 +08:00 |
|
Matei Zaharia
|
beb7ab8708
|
Merge pull request #467 from squito/executor_job_id
include jobid in Executor commandline args
|
2013-02-15 22:09:24 -08:00 |
|
haitao.yao
|
a9cfac347a
|
Merge branch 'mesos'
|
2013-02-16 10:11:28 +08:00 |
|
Tathagata Das
|
3bcc6e5c03
|
Merge pull request #466 from pwendell/java-stream-transform
STREAMING-50: Support transform workaround in JavaPairDStream
|
2013-02-14 21:30:55 -08:00 |
|