Tathagata Das
|
c6b2f765d3
|
Merge branch 'mesos-streaming' into streaming
|
2013-02-07 13:13:53 -08:00 |
|
Tathagata Das
|
12300758cc
|
Merge pull request #372 from Reinvigorate/sm-kafka
Removing offset management code that is non-existent in kafka 0.7.0+
|
2013-02-07 12:41:07 -08:00 |
|
Tathagata Das
|
915d9931fe
|
Merge pull request #373 from Reinvigorate/sm-updateStateByKey
StateDStream changes to give updateStateByKey consistent behavior
|
2013-02-07 11:59:19 -08:00 |
|
Tathagata Das
|
9c8ff1e55f
|
Fixed checkpoint testcases
|
2013-01-23 07:31:49 -08:00 |
|
Tathagata Das
|
666ce431aa
|
Added support for rescheduling unprocessed batches on master failure.
|
2013-01-23 03:15:36 -08:00 |
|
Tathagata Das
|
fad2b82fc8
|
Added support for saving input files of FileInputDStream to graph checkpoints. Modified 'file input stream with checkpoint' testcase to test recovery of pre-master-failure input files.
|
2013-01-22 18:10:00 -08:00 |
|
Tathagata Das
|
364cdb679c
|
Refactored DStreamCheckpointData.
|
2013-01-22 00:43:31 -08:00 |
|
seanm
|
c0694291c8
|
Splitting StreamingContext.queueStream into two methods
|
2013-01-20 12:09:45 -07:00 |
|
seanm
|
ea739251eb
|
adding updateStateByKey object lifecycle test
|
2013-01-20 11:29:21 -07:00 |
|
Tathagata Das
|
33bad85bb9
|
Fixed streaming testsuite bugs
|
2013-01-20 03:51:11 -08:00 |
|
Tathagata Das
|
4f8fe58b25
|
Merge branch 'mesos-streaming' into streaming
Conflicts:
core/src/main/scala/spark/api/java/JavaRDDLike.scala
core/src/main/scala/spark/api/java/JavaSparkContext.scala
core/src/test/scala/spark/JavaAPISuite.java
|
2013-01-20 01:13:56 -08:00 |
|
Tathagata Das
|
214345ceac
|
Fixed issue https://spark-project.atlassian.net/browse/STREAMING-29, along with updates to doc comments in SparkContext.checkpoint().
|
2013-01-19 23:50:17 -08:00 |
|
seanm
|
d3064fe707
|
kafkaStream API cleanup. A quorum of zookeepers can now be specified
|
2013-01-18 21:34:29 -07:00 |
|
seanm
|
56b7fbafa2
|
further KafkaInputDStream cleanup (removing unused and commented out code relating to offset management)
|
2013-01-18 21:15:54 -07:00 |
|
Patrick Wendell
|
c46dd2de78
|
Moving tests to appropriate directory
|
2013-01-17 21:43:17 -08:00 |
|
Patrick Wendell
|
e0165bf714
|
Adding queueStream and some slight refactoring
|
2013-01-17 21:25:49 -08:00 |
|
Patrick Wendell
|
ee0314c3b3
|
Merge branch 'streaming' into streaming-java-api
|
2013-01-17 18:43:00 -08:00 |
|
Patrick Wendell
|
70ba994d6d
|
Import fixup
|
2013-01-17 18:41:59 -08:00 |
|
Patrick Wendell
|
2261e62ee5
|
Style cleanup
|
2013-01-17 18:41:59 -08:00 |
|
Patrick Wendell
|
82b8707c6b
|
Checkpointing in Streaming java API
|
2013-01-17 18:41:58 -08:00 |
|
Patrick Wendell
|
61b877c688
|
Adding flatMap
|
2013-01-17 18:41:58 -08:00 |
|
Patrick Wendell
|
8e6cbbc6c7
|
Adding other updateState functions
|
2013-01-17 18:41:58 -08:00 |
|
Patrick Wendell
|
2a872335c5
|
Bug fix and test cleanup
|
2013-01-17 18:41:58 -08:00 |
|
seanm
|
a4639400ea
|
Merge branch 'streaming' into sm-updateStateByKey
|
2013-01-15 20:29:01 -07:00 |
|
Tathagata Das
|
1638fcb0dc
|
Fixed updateStateByKey to work with primitive types.
|
2013-01-14 17:18:39 -08:00 |
|
seanm
|
c203a29296
|
StateDStream changes to give updateStateByKey consistent behavior
|
2013-01-14 17:22:03 -07:00 |
|
seanm
|
b61a4ec773
|
Removing offset management code that is non-existent in kafka 0.7.0+
|
2013-01-14 17:13:10 -07:00 |
|
Patrick Wendell
|
a0013beb03
|
Stash
|
2013-01-14 15:15:02 -08:00 |
|
Patrick Wendell
|
8ad6220bd3
|
Bugfix
|
2013-01-14 14:56:36 -08:00 |
|
Patrick Wendell
|
38d9a3a863
|
Remove AnyRef constraint in updateState
|
2013-01-14 13:31:49 -08:00 |
|
Patrick Wendell
|
ae5290f4a2
|
Bug fix
|
2013-01-14 13:31:32 -08:00 |
|
Patrick Wendell
|
6069446356
|
Making comments consistent w/ Spark style
|
2013-01-14 10:42:05 -08:00 |
|
Patrick Wendell
|
d182a57cae
|
Two changes:
- Updating countByX() types based on bug fix
- Porting new documentation to Java
|
2013-01-14 10:03:55 -08:00 |
|
Patrick Wendell
|
a292ed8d8a
|
Some style cleanup
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
3461cd99b7
|
Flume example and bug fix
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
5bcb048167
|
More work on InputStreams
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
280b6d0186
|
Porting to new Duration class
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
c2537057f9
|
Fixing issue with <Long> types
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
b36c4f7cce
|
More work on StreamingContext
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
5004eec37c
|
Import Cleanup
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
2fe39a4468
|
Some docs for the JavaTestUtils
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
560c312c60
|
Docs, some tests, and work on
StreamingContext
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
7e1049d8f1
|
Squashing a few TODOs
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
74182010a4
|
Style cleanup and moving functions
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
056f5efc55
|
More pair functions
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
6e514a8d35
|
PairDStream and DStreamLike
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
f144e0413a
|
Adding transform and union
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
0d0bab25bd
|
Reduce tests
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
22a8c7be9a
|
Adding more tests
|
2013-01-14 09:42:36 -08:00 |
|
Patrick Wendell
|
91b3d41448
|
Better equality test (thanks Josh)
|
2013-01-14 09:42:36 -08:00 |
|