Tathagata Das
25fd684b89
Merge branch 'dev' of github.com:radlab/spark into dev
2012-09-04 20:44:14 -07:00
Tathagata Das
7c09ad0e04
Changed DStream member access permissions from private to protected. Updated StateDStream to checkpoint RDDs and forget lineage.
2012-09-04 19:11:49 -07:00
haoyuan
96a1f2277d
fix the compile error in TopKWordCountRaw.scala
2012-09-04 18:03:34 -07:00
haoyuan
2ff72f60ac
add TopKWordCountRaw.scala
2012-09-04 17:55:55 -07:00
Tathagata Das
389a78722c
Updated the return types of PairDStreamFunctions to return DStreams instead of ShuffleDStreams for cleaner abstraction.
2012-09-04 15:37:46 -07:00
root
7b892ee66e
Merge branch 'dev' of github.com:radlab/spark into dev
2012-09-04 04:27:10 +00:00
root
1878731671
Various test programs
2012-09-04 04:26:53 +00:00
Tathagata Das
b8e9e8ea78
Merge branch 'dev' of github.com:radlab/spark into dev
2012-09-02 02:35:32 -07:00
Tathagata Das
7419d2c7ea
Added transformRDD DStream operation and TransformedDStream. Added sbt assembly option for streaming project.
2012-09-02 02:35:17 -07:00
root
ceabf71257
tweaks
2012-09-01 21:52:42 +00:00
root
6025889be0
More raw network receiver programs
2012-09-01 20:51:07 +00:00
root
bf993cda63
Make batch size configurable in RawCount
2012-09-01 19:59:23 +00:00
root
83dad56334
Further fixes to raw text sender, plus an app that uses it
2012-09-01 19:45:25 +00:00
Matei Zaharia
f84d2bbe55
Bug fixes to RateLimitedOutputStream
2012-09-01 00:31:15 -07:00
Matei Zaharia
44758aa8e2
First work towards a RawInputDStream and a sender program for it.
2012-09-01 00:17:59 -07:00
Matei Zaharia
51fb13dd16
Bug fix
2012-08-31 15:36:11 -07:00
Matei Zaharia
ce42a46375
Bug fix
2012-08-31 15:35:35 -07:00
Matei Zaharia
f92d4a6ac1
Better output messages for streaming job duration
2012-08-31 15:33:48 -07:00
Tathagata Das
2d01d38a41
Added StateDStream, corresponding stateful stream operations, and testcases. Also refactored few PairDStreamFunctions methods.
2012-08-31 03:47:34 -07:00
root
e1da274a48
WordCount tweaks
2012-08-31 07:16:19 +00:00
root
d4d2cb670f
Make checkpoint interval configurable in WordCount2
2012-08-31 00:34:57 +00:00
root
1f8085b8d0
Compile fixes
2012-08-29 03:20:56 +00:00
Tathagata Das
43e66146f7
Merge branch 'dev' of github.com/radlab/spark into dev
2012-08-28 13:51:05 -07:00
Tathagata Das
b5b93a621c
Added capabllity to take streaming input from network. Renamed SparkStreamContext to StreamingContext.
2012-08-28 12:35:19 -07:00
root
e2cf197a0a
Made WordCount2 even more configurable
2012-08-27 03:34:15 +00:00
root
b78c5ae803
Merge branch 'dev' of github.com:radlab/spark into dev
2012-08-27 01:16:39 +00:00
root
9de1c3abf9
Tweaks to WordCount2
2012-08-27 00:57:00 +00:00
Matei Zaharia
57796b183e
Code style
2012-08-26 17:25:22 -07:00
Matei Zaharia
22b1a20e61
Made Time and Interval immutable
2012-08-26 17:04:34 -07:00
Matei Zaharia
23a29b6d19
Merge branch 'dev' of github.com:radlab/spark into dev
2012-08-26 16:45:37 -07:00
Matei Zaharia
b120e24fe0
Add equals and hashCode to Time
2012-08-26 16:45:14 -07:00
root
b08ff710af
Added sliding word count, and some fixes to reduce window DStream
2012-08-26 23:40:50 +00:00
Matei Zaharia
ad6537321e
Make Time serializable
2012-08-26 16:27:23 -07:00
Matei Zaharia
e7a5cbb543
Reduce log4j verbosity for streaming
2012-08-24 16:45:01 -07:00
Matei Zaharia
091b1438f5
Fix WordCount job name
2012-08-24 16:43:59 -07:00
Tathagata Das
cae894ee7a
Added new Clock interface that is used by RecurringTimer to scheduler events on system time or manually-configured time.
2012-08-06 14:52:46 -07:00
Matei Zaharia
43b81eb271
Renamed RDS to DStream, plus minor style fixes
2012-08-02 14:05:51 -04:00
Matei Zaharia
29bf44473c
Added an RDS that repeatedly returns the same input
2012-08-02 11:43:04 -04:00
Matei Zaharia
650d11817e
Added a WordCount for external data and fixed bugs in file streams
2012-08-02 11:09:43 -04:00
Tathagata Das
ed897ac5e1
Moved streaming files not immediately necessary to spark.streaming.util.
2012-08-01 22:28:54 -07:00
Tathagata Das
3be54c2a8a
1. Refactored SparkStreamContext, Scheduler, InputRDS, FileInputRDS and a few other files.
...
2. Modified Time class to represent milliseconds (long) directly, instead of LongTime.
3. Added new files QueueInputRDS, RecurringTimer, etc.
4. Added RDDSuite as the skeleton for testcases.
5. Added two examples in spark.streaming.examples.
6. Removed all past examples and a few unnecessary files. Moved a number of files to spark.streaming.util.
2012-08-01 22:09:27 -07:00
Tathagata Das
5a26ca4a80
Restructured file locations to separate examples and other programs from core programs.
2012-07-30 13:29:13 -07:00
Matei Zaharia
fcee4153b9
Renamed stream package to streaming
2012-07-29 13:35:22 -07:00
Matei Zaharia
47b7ebad12
Added the Spark Streaing code, ported to Akka 2
2012-07-28 20:03:26 -07:00
