Tathagata Das
|
d55e3aa467
|
Updated JavaStreamingContext with updated kafkaStream API.
|
2013-02-07 13:59:18 -08:00 |
|
Tathagata Das
|
c6b2f765d3
|
Merge branch 'mesos-streaming' into streaming
|
2013-02-07 13:13:53 -08:00 |
|
Tathagata Das
|
12300758cc
|
Merge pull request #372 from Reinvigorate/sm-kafka
Removing offset management code that is non-existent in kafka 0.7.0+
|
2013-02-07 12:41:07 -08:00 |
|
Tathagata Das
|
915d9931fe
|
Merge pull request #373 from Reinvigorate/sm-updateStateByKey
StateDStream changes to give updateStateByKey consistent behavior
|
2013-02-07 11:59:19 -08:00 |
|
Tathagata Das
|
666ce431aa
|
Added support for rescheduling unprocessed batches on master failure.
|
2013-01-23 03:15:36 -08:00 |
|
Tathagata Das
|
fad2b82fc8
|
Added support for saving input files of FileInputDStream to graph checkpoints. Modified 'file input stream with checkpoint' testcase to test recovery of pre-master-failure input files.
|
2013-01-22 18:10:00 -08:00 |
|
Tathagata Das
|
364cdb679c
|
Refactored DStreamCheckpointData.
|
2013-01-22 00:43:31 -08:00 |
|
seanm
|
c0694291c8
|
Splitting StreamingContext.queueStream into two methods
|
2013-01-20 12:09:45 -07:00 |
|
Tathagata Das
|
4f8fe58b25
|
Merge branch 'mesos-streaming' into streaming
Conflicts:
core/src/main/scala/spark/api/java/JavaRDDLike.scala
core/src/main/scala/spark/api/java/JavaSparkContext.scala
core/src/test/scala/spark/JavaAPISuite.java
|
2013-01-20 01:13:56 -08:00 |
|
Tathagata Das
|
214345ceac
|
Fixed issue https://spark-project.atlassian.net/browse/STREAMING-29, along with updates to doc comments in SparkContext.checkpoint().
|
2013-01-19 23:50:17 -08:00 |
|
seanm
|
d3064fe707
|
kafkaStream API cleanup. A quorum of zookeepers can now be specified
|
2013-01-18 21:34:29 -07:00 |
|
seanm
|
56b7fbafa2
|
further KafkaInputDStream cleanup (removing unused and commented out code relating to offset management)
|
2013-01-18 21:15:54 -07:00 |
|
Patrick Wendell
|
e0165bf714
|
Adding queueStream and some slight refactoring
|
2013-01-17 21:25:49 -08:00 |
|
Patrick Wendell
|
70ba994d6d
|
Import fixup
|
2013-01-17 18:41:59 -08:00 |
|
Patrick Wendell
|
2261e62ee5
|
Style cleanup
|
2013-01-17 18:41:59 -08:00 |
|
Patrick Wendell
|
82b8707c6b
|
Checkpointing in Streaming java API
|
2013-01-17 18:41:58 -08:00 |
|
Patrick Wendell
|
61b877c688
|
Adding flatMap
|
2013-01-17 18:41:58 -08:00 |
|
Patrick Wendell
|
8e6cbbc6c7
|
Adding other updateState functions
|
2013-01-17 18:41:58 -08:00 |
|
Patrick Wendell
|
2a872335c5
|
Bug fix and test cleanup
|
2013-01-17 18:41:58 -08:00 |
|
seanm
|
a4639400ea
|
Merge branch 'streaming' into sm-updateStateByKey
|
2013-01-15 20:29:01 -07:00 |
|
Tathagata Das
|
1638fcb0dc
|
Fixed updateStateByKey to work with primitive types.
|
2013-01-14 17:18:39 -08:00 |
|
seanm
|
c203a29296
|
StateDStream changes to give updateStateByKey consistent behavior
|
2013-01-14 17:22:03 -07:00 |
|
seanm
|
b61a4ec773
|
Removing offset management code that is non-existent in kafka 0.7.0+
|
2013-01-14 17:13:10 -07:00 |
|
Patrick Wendell
|
a0013beb03
|
Stash
|
2013-01-14 15:15:02 -08:00 |
|
Patrick Wendell
|
8ad6220bd3
|
Bugfix
|
2013-01-14 14:56:36 -08:00 |
|
Patrick Wendell
|
38d9a3a863
|
Remove AnyRef constraint in updateState
|
2013-01-14 13:31:49 -08:00 |
|
Patrick Wendell
|
ae5290f4a2
|
Bug fix
|
2013-01-14 13:31:32 -08:00 |
|
Patrick Wendell
|
6069446356
|
Making comments consistent w/ Spark style
|
2013-01-14 10:42:05 -08:00 |
|
Patrick Wendell
|
d182a57cae
|
Two changes:
- Updating countByX() types based on bug fix
- Porting new documentation to Java
|
2013-01-14 10:03:55 -08:00 |
|
Patrick Wendell
|
a292ed8d8a
|
Some style cleanup
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
3461cd99b7
|
Flume example and bug fix
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
5bcb048167
|
More work on InputStreams
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
280b6d0186
|
Porting to new Duration class
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
c2537057f9
|
Fixing issue with <Long> types
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
b36c4f7cce
|
More work on StreamingContext
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
5004eec37c
|
Import Cleanup
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
560c312c60
|
Docs, some tests, and work on
StreamingContext
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
7e1049d8f1
|
Squashing a few TODOs
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
74182010a4
|
Style cleanup and moving functions
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
056f5efc55
|
More pair functions
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
6e514a8d35
|
PairDStream and DStreamLike
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
f144e0413a
|
Adding transform and union
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
22a8c7be9a
|
Adding more tests
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
867a7455e2
|
Adding some initial tests to streaming API.
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
b607c9e916
|
A very rough, early cut at some Java functionality for Streaming.
|
2013-01-14 09:42:36 -08:00 |
|
Tathagata Das
|
f90f794cde
|
Minor name fix
|
2013-01-13 21:25:57 -08:00 |
|
Tathagata Das
|
0dbd411a56
|
Added documentation for PairDStreamFunctions.
|
2013-01-13 21:08:35 -08:00 |
|
Tathagata Das
|
0a2e333341
|
Removed stream id from the constructor of NetworkReceiver to make it easier for PluggableNetworkInputDStream.
|
2013-01-13 16:18:39 -08:00 |
|
Tathagata Das
|
365506fb03
|
Changed variable name form ***Time to ***Duration to keep things consistent.
|
2013-01-09 14:29:25 -08:00 |
|
Tathagata Das
|
156e8b47ef
|
Split Time to Time (absolute instant of time) and Duration (duration of time).
|
2013-01-09 12:42:10 -08:00 |
|