Denny
|
1588d4dbe6
|
Renamed class.
|
2012-09-04 21:13:25 -07:00 |
|
|
Denny
|
22dde6e020
|
Start a standalone cluster locally.
|
2012-09-04 20:56:30 -07:00 |
|
Tathagata Das
|
25fd684b89
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-04 20:44:14 -07:00 |
|
|
Tathagata Das
|
7c09ad0e04
|
Changed DStream member access permissions from private to protected. Updated StateDStream to checkpoint RDDs and forget lineage.
|
2012-09-04 19:11:49 -07:00 |
|
haoyuan
|
96a1f2277d
|
fix the compile error in TopKWordCountRaw.scala
|
2012-09-04 18:03:34 -07:00 |
|
|
haoyuan
|
2ff72f60ac
|
add TopKWordCountRaw.scala
|
2012-09-04 17:55:55 -07:00 |
|
|
Tathagata Das
|
389a78722c
|
Updated the return types of PairDStreamFunctions to return DStreams instead of ShuffleDStreams for cleaner abstraction.
|
2012-09-04 15:37:46 -07:00 |
|
root
|
7b892ee66e
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-04 04:27:10 +00:00 |
|
|
root
|
1878731671
|
Various test programs
|
2012-09-04 04:26:53 +00:00 |
|
Matei Zaharia
|
a842c63044
|
Minor formatting fixes
|
2012-09-03 16:24:00 -07:00 |
|
Mosharaf Chowdhury
|
5e8e8e4c9d
|
Merge remote-tracking branch 'upstream/dev' into dev
|
2012-09-03 16:16:56 -07:00 |
|
|
Matei Zaharia
|
2d6a629f8c
|
Merge pull request #182 from HarveyFeng/dev-fetch
Add a limit on the number of parallel fetches in the reduce stage
|
2012-09-03 16:14:57 -07:00 |
|
|
Tathagata Das
|
b8e9e8ea78
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-02 02:35:32 -07:00 |
|
|
Tathagata Das
|
7419d2c7ea
|
Added transformRDD DStream operation and TransformedDStream. Added sbt assembly option for streaming project.
|
2012-09-02 02:35:17 -07:00 |
|
root
|
ceabf71257
|
tweaks
|
2012-09-01 21:52:42 +00:00 |
|
|
root
|
6025889be0
|
More raw network receiver programs
|
2012-09-01 20:51:07 +00:00 |
|
|
root
|
bf993cda63
|
Make batch size configurable in RawCount
|
2012-09-01 19:59:23 +00:00 |
|
|
root
|
83dad56334
|
Further fixes to raw text sender, plus an app that uses it
|
2012-09-01 19:45:25 +00:00 |
|
Harvey
|
3076b038f4
|
Start fetching a remote block when a received remote block has been passed
to the reduce function
|
2012-09-01 12:01:35 -07:00 |
|
|
Mosharaf Chowdhury
|
10dfee7501
|
Merge remote-tracking branch 'upstream/dev' into dev
|
2012-09-01 00:33:00 -07:00 |
|
|
Matei Zaharia
|
f84d2bbe55
|
Bug fixes to RateLimitedOutputStream
|
2012-09-01 00:31:15 -07:00 |
|
|
Matei Zaharia
|
44758aa8e2
|
First work towards a RawInputDStream and a sender program for it.
|
2012-09-01 00:17:59 -07:00 |
|
root
|
c42e7ac282
|
More block manager fixes
|
2012-09-01 04:31:11 +00:00 |
|
|
Matei Zaharia
|
389fb4cc54
|
End runJob() with a SparkException when a task fails too many times in
one of the cluster schedulers.
|
2012-08-31 17:47:43 -07:00 |
|
|
Matei Zaharia
|
51fb13dd16
|
Bug fix
|
2012-08-31 15:36:11 -07:00 |
|
|
Matei Zaharia
|
ce42a46375
|
Bug fix
|
2012-08-31 15:35:35 -07:00 |
|
|
Matei Zaharia
|
f92d4a6ac1
|
Better output messages for streaming job duration
|
2012-08-31 15:33:48 -07:00 |
|
|
Matei Zaharia
|
607b8fffcd
|
End runJob with a SparkException when a Mesos task fails too many times
|
2012-08-31 11:40:12 -07:00 |
|
|
Tathagata Das
|
2d01d38a41
|
Added StateDStream, corresponding stateful stream operations, and testcases. Also refactored few PairDStreamFunctions methods.
|
2012-08-31 03:47:34 -07:00 |
|
root
|
e1da274a48
|
WordCount tweaks
|
2012-08-31 07:16:19 +00:00 |
|
|
root
|
113277549c
|
Really fixed the replication-3 issue. The problem was a few buffers not being rewound.
|
2012-08-31 05:39:35 +00:00 |
|
|
Mosharaf Chowdhury
|
baf2a7ccd2
|
Merge remote-tracking branch 'upstream/dev' into dev
|
2012-08-30 22:28:14 -07:00 |
|
|
Mosharaf Chowdhury
|
31ffe8d528
|
Synchronization bug fix in broadcast implementations
|
2012-08-30 22:26:43 -07:00 |
|
|
Matei Zaharia
|
101ae493e2
|
Replicate serialized blocks properly, without sharing a ByteBuffer.
|
2012-08-30 22:24:14 -07:00 |
|
|
Mosharaf Chowdhury
|
3883532545
|
Bug fix. Fixed log messages. Updated BroadcastTest example to have iterations.
|
2012-08-30 21:43:00 -07:00 |
|
|
Matei Zaharia
|
a480dec6b2
|
Deserialize multi-get results in the caller's thread. This fixes an
issue with shared buffers in the KryoSerializer.
|
2012-08-30 20:01:06 -07:00 |
|
|
Matei Zaharia
|
1b3e3352eb
|
Deserialize multi-get results in the caller's thread. This fixes an
issue with shared buffers with the KryoSerializer.
|
2012-08-30 17:59:25 -07:00 |
|
|
root
|
d4d2cb670f
|
Make checkpoint interval configurable in WordCount2
|
2012-08-31 00:34:57 +00:00 |
|
|
root
|
c4366eb764
|
Fixes to ShuffleFetcher
|
2012-08-31 00:34:24 +00:00 |
|
|
Mosharaf Chowdhury
|
8f2bd399da
|
Merge remote-tracking branch 'upstream/dev' into dev
|
2012-08-30 15:21:08 -07:00 |
|
|
Matei Zaharia
|
bf3212615a
|
Merge pull request #184 from rxin/dev
Disable running combiners on map tasks when mergeCombiners function is not specified by the user.
|
2012-08-30 14:12:40 -07:00 |
|
Reynold Xin
|
a8a2a08a1a
|
Added a test for testing map-side combine on/off switch.
|
2012-08-30 12:34:28 -07:00 |
|
|
Matei Zaharia
|
62e5326af0
|
Wording
|
2012-08-30 08:37:43 -07:00 |
|
|
Matei Zaharia
|
e8ac9221dc
|
Update sbt build command to create JARs
|
2012-08-30 08:36:39 -07:00 |
|
|
Reynold Xin
|
5945bcdcc5
|
Added a new flag in Aggregator to indicate applying map side combiners.
|
2012-08-29 23:32:08 -07:00 |
|
|
Reynold Xin
|
c68e820b2a
|
Merge branch 'dev' of github.com:mesos/spark into dev
|
2012-08-29 23:01:19 -07:00 |
|
|
Reynold Xin
|
940869dfda
|
Disable running combiners on map tasks when mergeCombiners function is
not specified by the user.
|
2012-08-29 23:00:02 -07:00 |
|
|
Tathagata Das
|
4db3a96766
|
Made minor changes to reduce compilation errors in Eclipse. Twirl stuff still does not compile in Eclipse.
|
2012-08-29 13:04:01 -07:00 |
|
|
Matei Zaharia
|
84bf7924d6
|
Made region used by spark-ec2 configurable.
|
2012-08-28 22:40:48 -07:00 |
|
|
Matei Zaharia
|
47507d69d9
|
Made region used by spark-ec2 configurable.
|
2012-08-28 22:40:00 -07:00 |
|