Patrick Wendell
ee0314c3b3
Merge branch 'streaming' into streaming-java-api
2013-01-17 18:43:00 -08:00
Patrick Wendell
70ba994d6d
Import fixup
2013-01-17 18:41:59 -08:00
Patrick Wendell
2261e62ee5
Style cleanup
2013-01-17 18:41:59 -08:00
Patrick Wendell
82b8707c6b
Checkpointing in Streaming java API
2013-01-17 18:41:58 -08:00
Patrick Wendell
61b877c688
Adding flatMap
2013-01-17 18:41:58 -08:00
Patrick Wendell
8e6cbbc6c7
Adding other updateState functions
2013-01-17 18:41:58 -08:00
Patrick Wendell
2a872335c5
Bug fix and test cleanup
2013-01-17 18:41:58 -08:00
Tathagata Das
1638fcb0dc
Fixed updateStateByKey to work with primitive types.
2013-01-14 17:18:39 -08:00
Patrick Wendell
a0013beb03
Stash
2013-01-14 15:15:02 -08:00
Patrick Wendell
8ad6220bd3
Bugfix
2013-01-14 14:56:36 -08:00
Patrick Wendell
38d9a3a863
Remove AnyRef constraint in updateState
2013-01-14 13:31:49 -08:00
Patrick Wendell
ae5290f4a2
Bug fix
2013-01-14 13:31:32 -08:00
Patrick Wendell
6069446356
Making comments consistent w/ Spark style
2013-01-14 10:42:05 -08:00
Patrick Wendell
d182a57cae
Two changes:
...
- Updating countByX() types based on bug fix
- Porting new documentation to Java
2013-01-14 10:03:55 -08:00
Patrick Wendell
a292ed8d8a
Some style cleanup
2013-01-14 09:42:36 -08:00
Patrick Wendell
3461cd99b7
Flume example and bug fix
2013-01-14 09:42:36 -08:00
Patrick Wendell
5bcb048167
More work on InputStreams
2013-01-14 09:42:36 -08:00
Patrick Wendell
280b6d0186
Porting to new Duration class
2013-01-14 09:42:36 -08:00
Patrick Wendell
c2537057f9
Fixing issue with <Long> types
2013-01-14 09:42:36 -08:00
Patrick Wendell
b36c4f7cce
More work on StreamingContext
2013-01-14 09:42:36 -08:00
Patrick Wendell
5004eec37c
Import Cleanup
2013-01-14 09:42:36 -08:00
Patrick Wendell
2fe39a4468
Some docs for the JavaTestUtils
2013-01-14 09:42:36 -08:00
Patrick Wendell
560c312c60
Docs, some tests, and work on
...
StreamingContext
2013-01-14 09:42:36 -08:00
Patrick Wendell
7e1049d8f1
Squashing a few TODOs
2013-01-14 09:42:36 -08:00
Patrick Wendell
74182010a4
Style cleanup and moving functions
2013-01-14 09:42:36 -08:00
Patrick Wendell
056f5efc55
More pair functions
2013-01-14 09:42:36 -08:00
Patrick Wendell
6e514a8d35
PairDStream and DStreamLike
2013-01-14 09:42:36 -08:00
Patrick Wendell
f144e0413a
Adding transform and union
2013-01-14 09:42:36 -08:00
Patrick Wendell
0d0bab25bd
Reduce tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
22a8c7be9a
Adding more tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
91b3d41448
Better equality test (thanks Josh)
2013-01-14 09:42:36 -08:00
Patrick Wendell
b0974e6c1d
Remving main method from tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
867a7455e2
Adding some initial tests to streaming API.
2013-01-14 09:42:36 -08:00
Patrick Wendell
b607c9e916
A very rough, early cut at some Java functionality for Streaming.
2013-01-14 09:42:36 -08:00
Tathagata Das
f90f794cde
Minor name fix
2013-01-13 21:25:57 -08:00
Tathagata Das
6cc8592f26
Fixed bug
2013-01-13 21:20:49 -08:00
Tathagata Das
0dbd411a56
Added documentation for PairDStreamFunctions.
2013-01-13 21:08:35 -08:00
Tathagata Das
0a2e333341
Removed stream id from the constructor of NetworkReceiver to make it easier for PluggableNetworkInputDStream.
2013-01-13 16:18:39 -08:00
Tathagata Das
365506fb03
Changed variable name form ***Time to ***Duration to keep things consistent.
2013-01-09 14:29:25 -08:00
Tathagata Das
156e8b47ef
Split Time to Time (absolute instant of time) and Duration (duration of time).
2013-01-09 12:42:10 -08:00
Tathagata Das
8c1b872512
Moved Twitter example to the where the other examples are.
2013-01-07 17:48:10 -08:00
Tathagata Das
64dceec293
Merge branch 'streaming-merge' into dev-merge
2013-01-07 16:54:35 -08:00
Tathagata Das
d808e1026a
Merge branch 'dev' into dev-merge
2013-01-07 16:41:11 -08:00
Tathagata Das
4719e6d8fe
Changed locations for unit test logs.
2013-01-07 16:06:07 -08:00
Tathagata Das
e60514d79e
Fixed bug
2013-01-07 15:16:16 -08:00
Tathagata Das
237bac36e9
Renamed examples and added documentation.
2013-01-07 14:37:21 -08:00
Tathagata Das
af8738dfb5
Moved Spark Streaming examples to examples sub-project.
2013-01-06 19:31:54 -08:00
Patrick Wendell
2ef993d159
BufferingBlockCreator -> NetworkReceiver.BlockGenerator
2013-01-02 14:19:51 -08:00
Patrick Wendell
96a6ff0b09
Merge branch 'dev-merge' into datahandler-fix
...
Conflicts:
streaming/src/main/scala/spark/streaming/dstream/DataHandler.scala
2013-01-02 14:08:15 -08:00
Patrick Wendell
493d65ce65
Several code-quality improvements to DataHandler.
...
- Changed to more accurate name: BufferingBlockCreator
- Docstring now correctly reflects the abstraction
offered by the class
- Made internal methods private
- Fixed indentation problems
2013-01-02 13:39:18 -08:00