seanm
a4639400ea
Merge branch 'streaming' into sm-updateStateByKey
2013-01-15 20:29:01 -07:00
Tathagata Das
eded21925a
Merge pull request #375 from tdas/streaming
...
Important bug fixes
2013-01-14 23:06:40 -08:00
Tathagata Das
1638fcb0dc
Fixed updateStateByKey to work with primitive types.
2013-01-14 17:18:39 -08:00
seanm
c203a29296
StateDStream changes to give updateStateByKey consistent behavior
2013-01-14 17:22:03 -07:00
Tathagata Das
131be5d62e
Fixed bug in RDD checkpointing.
2013-01-14 03:28:25 -08:00
Tathagata Das
82b0cc90ca
Merge pull request #370 from tdas/streaming
...
Added more documentation and minor change in API for NetworkReceiver
2013-01-13 21:28:12 -08:00
Tathagata Das
f90f794cde
Minor name fix
2013-01-13 21:25:57 -08:00
Tathagata Das
6cc8592f26
Fixed bug
2013-01-13 21:20:49 -08:00
Tathagata Das
0dbd411a56
Added documentation for PairDStreamFunctions.
2013-01-13 21:08:35 -08:00
Tathagata Das
0a2e333341
Removed stream id from the constructor of NetworkReceiver to make it easier for PluggableNetworkInputDStream.
2013-01-13 16:18:39 -08:00
Tathagata Das
23b54f62b7
Merge pull request #366 from tdas/streaming
...
Code refactoring
2013-01-13 14:07:34 -08:00
Matei Zaharia
530493b0e8
Merge pull request #365 from ryanlecompte/rate_limiter_timing_cleanup
...
Improve sleeping algorithm for rate limiting output streams
2013-01-13 10:47:00 -08:00
Ryan LeCompte
c31931af7e
switch to uppercase constants
2013-01-13 10:39:47 -08:00
Ryan LeCompte
2305a2c1d9
more code cleanup
2013-01-13 10:01:56 -08:00
Ryan LeCompte
addff2c466
add comment
2013-01-12 09:57:29 -08:00
Ryan LeCompte
ea20ae6618
add one extra test
2013-01-12 09:18:00 -08:00
Ryan LeCompte
2c77eeebb6
correct test params
2013-01-12 00:13:45 -08:00
Ryan LeCompte
0cfea7a2ec
add unit test
2013-01-11 23:48:07 -08:00
Ryan LeCompte
ff10b3aa09
add missing return
2013-01-11 21:03:57 -08:00
Ryan LeCompte
22445fbea9
attempt to sleep for more accurate time period, minor cleanup
2013-01-11 13:30:49 -08:00
Tathagata Das
365506fb03
Changed variable name form ***Time to ***Duration to keep things consistent.
2013-01-09 14:29:25 -08:00
Tathagata Das
156e8b47ef
Split Time to Time (absolute instant of time) and Duration (duration of time).
2013-01-09 12:42:10 -08:00
Patrick Wendell
6c502e3793
Making the Twitter example distributed.
...
This adds a distributed (receiver-based) implementation of the
Twitter dstream. It also changes the example to perform a
distributed sort rather than collecting the dataset at one node.
2013-01-07 22:01:11 -08:00
Tathagata Das
8c1b872512
Moved Twitter example to the where the other examples are.
2013-01-07 17:48:10 -08:00
Matei Zaharia
f7cf035b9b
Merge pull request #350 from tdas/streaming
...
Spark Streaming
2013-01-07 17:40:11 -08:00
Tathagata Das
64dceec293
Merge branch 'streaming-merge' into dev-merge
2013-01-07 16:54:35 -08:00
Tathagata Das
d808e1026a
Merge branch 'dev' into dev-merge
2013-01-07 16:41:11 -08:00
Tathagata Das
1d8b1c9bec
Merge branch 'dev-merge' of github.com:radlab/spark into dev-merge
2013-01-07 16:14:11 -08:00
Tathagata Das
4719e6d8fe
Changed locations for unit test logs.
2013-01-07 16:06:07 -08:00
Tathagata Das
e60514d79e
Fixed bug
2013-01-07 15:16:16 -08:00
Tathagata Das
3b0a3b89ac
Added better docs for RDDCheckpointData
2013-01-07 14:55:49 -08:00
Tathagata Das
237bac36e9
Renamed examples and added documentation.
2013-01-07 14:37:21 -08:00
Tathagata Das
1346126485
Changed cleanup to clearOldValues for TimeStampedHashMap and TimeStampedHashSet.
2013-01-07 12:11:27 -08:00
Tathagata Das
af8738dfb5
Moved Spark Streaming examples to examples sub-project.
2013-01-06 19:31:54 -08:00
Tathagata Das
934ecc829a
Removed streaming-env.sh.template
2013-01-06 14:15:07 -08:00
Matei Zaharia
ecf9c08901
Fix Accumulators in Java, and add a test for them
2013-01-05 20:54:08 -05:00
Patrick Wendell
c438faeac4
Merge pull request #10 from radlab/datahandler-fix
...
Several code-quality improvements to DataHandler.
2013-01-02 17:07:12 -08:00
Patrick Wendell
2ef993d159
BufferingBlockCreator -> NetworkReceiver.BlockGenerator
2013-01-02 14:19:51 -08:00
Patrick Wendell
96a6ff0b09
Merge branch 'dev-merge' into datahandler-fix
...
Conflicts:
streaming/src/main/scala/spark/streaming/dstream/DataHandler.scala
2013-01-02 14:08:15 -08:00
Patrick Wendell
493d65ce65
Several code-quality improvements to DataHandler.
...
- Changed to more accurate name: BufferingBlockCreator
- Docstring now correctly reflects the abstraction
offered by the class
- Made internal methods private
- Fixed indentation problems
2013-01-02 13:39:18 -08:00
Tathagata Das
3dc87dd923
Fixed compilation bug in RDDSuite created during merge for mesos/master.
2013-01-01 16:38:04 -08:00
Tathagata Das
d34dba25c2
Merge branch 'mesos' into dev-merge
2013-01-01 15:48:39 -08:00
Tathagata Das
02497f0cd4
Updated Streaming Programming Guide.
2013-01-01 12:21:32 -08:00
Matei Zaharia
55809fbc6d
Merge pull request #349 from woggling/cache-finally
...
Avoid stalls when computation of cached RDD throws exception
2013-01-01 08:21:33 -08:00
Matei Zaharia
c593f6329e
Merge pull request #348 from JoshRosen/spark-597
...
Raise exception when hashing Java arrays (SPARK-597)
2013-01-01 08:20:06 -08:00
Charles Reiss
58072a7340
Remove some dead comments
2013-01-01 08:07:44 -08:00
Charles Reiss
21636ee4fa
Test with exception while computing cached RDD.
2013-01-01 08:07:40 -08:00
Charles Reiss
feadaf72f4
Mark key as not loading in CacheTracker even when compute() fails
2013-01-01 07:57:20 -08:00
Josh Rosen
f803953998
Raise exception when hashing Java arrays (SPARK-597)
2012-12-31 20:20:11 -08:00
Tathagata Das
18b9b3b99f
More classes made private[streaming] to hide from scala docs.
2012-12-30 20:00:42 -08:00
