Patrick Wendell
|
f51b0f93f2
|
Adding Java-accessible methods to Vector.scala
This is needed for the Strata machine learning tutorial (and
also is generally helpful).
|
2013-02-23 13:26:59 -08:00 |
|
Matei Zaharia
|
d942d39072
|
Handle exceptions in RecordReader.close() better (suggested by Jim
Donahue)
|
2013-02-23 11:19:07 -08:00 |
|
Matei Zaharia
|
c89824046a
|
Merge pull request #490 from woggling/conn-death
Detect when SendingConnections disconnect even if we aren't sending to them
|
2013-02-22 22:58:19 -08:00 |
|
Tathagata Das
|
d8cee52d52
|
Merge branch 'mesos-streaming' into streaming
|
2013-02-22 18:25:34 -08:00 |
|
Tathagata Das
|
24c0cd6168
|
Fixed resolver for akka-zeromq
|
2013-02-22 18:23:29 -08:00 |
|
Charles Reiss
|
50cf8c8b79
|
Add fault tolerance test that uses replicated RDDs.
|
2013-02-22 16:11:53 -08:00 |
|
Charles Reiss
|
c8a7886921
|
Detect when SendingConnections drop by trying to read them.
Comment fix
|
2013-02-22 16:11:52 -08:00 |
|
Matei Zaharia
|
d4d7993bf5
|
Several fixes to the work to log when no resources can be used by a job.
Fixed some of the messages as well as code style.
|
2013-02-22 15:51:37 -08:00 |
|
Matei Zaharia
|
f33662c133
|
Merge remote-tracking branch 'pwendell/starvation-check'
Also fixed a bug where master was offering executors on dead workers
Conflicts:
core/src/main/scala/spark/deploy/master/Master.scala
|
2013-02-22 15:27:41 -08:00 |
|
Matei Zaharia
|
7341de0d48
|
Merge pull request #475 from JoshRosen/spark-668
Remove hack workaround for SPARK-668
|
2013-02-22 14:56:18 -08:00 |
|
Matei Zaharia
|
fcf7fa8448
|
Merge pull request #489 from pwendell/SPARK-702
SPARK-702: Replace Function --> JFunction in JavaAPI Suite.
|
2013-02-22 14:55:46 -08:00 |
|
Patrick Wendell
|
f8c3a03d55
|
SPARK-702: Replace Function --> JFunction in JavaAPI Suite.
In a few places the Scala (rather than Java) function class is used.
|
2013-02-22 12:54:15 -08:00 |
|
Tathagata Das
|
cfa65ebff1
|
Merge pull request #480 from MLnick/streaming-eg-algebird
[Streaming] Examples using Twitter's Algebird library
|
2013-02-22 12:29:04 -08:00 |
|
Tathagata Das
|
688e62718f
|
Merge pull request #479 from ScrapCodes/zeromq-streaming
Zeromq streaming
|
2013-02-22 12:17:17 -08:00 |
|
Imran Rashid
|
0f37b43b40
|
make the ShuffleFetcher responsible for collecting shuffle metrics, which gives us metrics for CoGroupedRDD and ShuffledRDD
|
2013-02-21 16:56:28 -08:00 |
|
Imran Rashid
|
9230617f23
|
add cleanup iterator
|
2013-02-21 16:55:14 -08:00 |
|
Tathagata Das
|
208edaac1b
|
Fixed condition in InputDStream isTimeValid.
|
2013-02-21 15:22:26 -08:00 |
|
Imran Rashid
|
81bd07da26
|
sparkListeners should be a val
|
2013-02-21 15:21:45 -08:00 |
|
Imran Rashid
|
796e934d31
|
add some docs & some cleanup
|
2013-02-21 15:19:34 -08:00 |
|
Imran Rashid
|
394d3acc3e
|
store taskInfo & metrics together in a tuple
|
2013-02-21 15:19:34 -08:00 |
|
Imran Rashid
|
7960927cf4
|
get rid of a bunch of boilerplate; more formatting happens in Listener, not StageInfo
|
2013-02-21 15:19:34 -08:00 |
|
Imran Rashid
|
d0bfac3eed
|
taskInfo tracks if a task is run on a preferred host
|
2013-02-21 15:19:34 -08:00 |
|
Imran Rashid
|
6f62a57858
|
add runtime breakdowns
|
2013-02-21 15:19:34 -08:00 |
|
Imran Rashid
|
176cb20703
|
add task result size; better formatting for time interval distributions; cleanup distribution formatting
|
2013-02-21 15:19:33 -08:00 |
|
Imran Rashid
|
f2fcabf2ea
|
add timing around parts of executor & track result size
|
2013-02-21 15:19:33 -08:00 |
|
Imran Rashid
|
ff127cfcd3
|
Merge branch 'master' into stageInfo
Conflicts:
core/src/main/scala/spark/SparkContext.scala
core/src/main/scala/spark/storage/BlockManager.scala
|
2013-02-21 15:16:21 -08:00 |
|
Imran Rashid
|
69f9a7035f
|
fully revert change to addOnCompleteCallback -- missed this in e9f53ec
|
2013-02-21 15:07:46 -08:00 |
|
Imran Rashid
|
baab23abdf
|
TaskContext does not hold a reference to Task; instead, it has a shared instance of TaskMetrics with Task
|
2013-02-21 14:13:01 -08:00 |
|
Nick Pentreath
|
d9bdae8cc2
|
Adding documentation for HLL and CMS examples. More efficient and clear use of the monoids.
|
2013-02-21 12:31:31 +02:00 |
|
Nick Pentreath
|
718474b9c6
|
Bumping Algebird to 0.1.9
|
2013-02-21 12:11:31 +02:00 |
|
Nick Pentreath
|
16d456742e
|
Merge remote-tracking branch 'upstream/streaming' into streaming-eg-algebird
|
2013-02-21 09:33:08 +02:00 |
|
haitao.yao
|
8215b95547
|
Merge branch 'mesos'
|
2013-02-21 10:07:24 +08:00 |
|
Tathagata Das
|
2921fa7d81
|
Merge pull request #487 from tdas/streaming
Merged latest master changes to streaming
|
2013-02-20 11:06:57 -08:00 |
|
Tathagata Das
|
972fe7714f
|
Merge branch 'mesos-streaming' into streaming
Conflicts:
streaming/src/test/java/spark/streaming/JavaAPISuite.java
|
2013-02-20 11:06:01 -08:00 |
|
Tathagata Das
|
334ab92441
|
Fixed bug in CheckpointSuite
|
2013-02-20 10:26:36 -08:00 |
|
Tathagata Das
|
1cb725e417
|
Merge branch 'mesos-master' into streaming
|
2013-02-20 09:55:35 -08:00 |
|
Tathagata Das
|
fb9956256d
|
Merge branch 'mesos-master' into streaming
Conflicts:
core/src/main/scala/spark/rdd/CheckpointRDD.scala
streaming/src/main/scala/spark/streaming/dstream/ReducedWindowedDStream.scala
|
2013-02-20 09:01:29 -08:00 |
|
Prashant Sharma
|
ee885611ff
|
removed document on zeromq
|
2013-02-20 19:18:24 +05:30 |
|
Prashant Sharma
|
4e5b09664c
|
fixes corresponding to review feedback at pull request #479
|
2013-02-20 19:14:52 +05:30 |
|
Tathagata Das
|
991d3342fe
|
Merge pull request #486 from ScrapCodes/akka-example-bug-fix
A bug fix to AkkaWordCount example.
|
2013-02-20 02:06:36 -08:00 |
|
Prashant Sharma
|
05dc385649
|
A bug fix post merge, following changes to AkkaUtils
|
2013-02-20 15:28:12 +05:30 |
|
Matei Zaharia
|
05bc02e80b
|
Merge pull request #482 from woggling/shutdown-exceptions
Don't call System.exit over uncaught exceptions from shutdown hooks
|
2013-02-19 20:56:15 -08:00 |
|
haitao.yao
|
6a3d44c673
|
Merge branch 'mesos'
|
2013-02-20 10:23:58 +08:00 |
|
Charles Reiss
|
092c631fa8
|
Pull detection of being in a shutdown hook into utility function.
|
2013-02-19 17:49:55 -08:00 |
|
Matei Zaharia
|
8a992226bd
|
Merge pull request #484 from andyk/master
Fixes a broken link in documentation to issue tracker
|
2013-02-19 17:07:24 -08:00 |
|
Matei Zaharia
|
a3e86b2b1f
|
Merge pull request #483 from rxin/splitpruningrdd2
Added a method to create PartitionPruningRDD.
|
2013-02-19 17:07:00 -08:00 |
|
Andy Konwinski
|
ecd137a72d
|
Fixes link to issue tracker in documentation page "Contributing to Spark".
|
2013-02-19 16:58:02 -08:00 |
|
Reynold Xin
|
130f704baf
|
Added a method to create PartitionPruningRDD.
|
2013-02-19 16:03:52 -08:00 |
|
Charles Reiss
|
d0588bd6d7
|
Catch/log errors deleting temp dirs
|
2013-02-19 13:04:06 -08:00 |
|
Charles Reiss
|
687581c3ec
|
Paranoid uncaught exception handling for exceptions during shutdown
|
2013-02-19 13:03:02 -08:00 |
|