Patrick Wendell
5bcb048167
More work on InputStreams
2013-01-14 09:42:36 -08:00
Patrick Wendell
280b6d0186
Porting to new Duration class
2013-01-14 09:42:36 -08:00
Patrick Wendell
c2537057f9
Fixing issue with <Long> types
2013-01-14 09:42:36 -08:00
Patrick Wendell
b36c4f7cce
More work on StreamingContext
2013-01-14 09:42:36 -08:00
Patrick Wendell
5004eec37c
Import Cleanup
2013-01-14 09:42:36 -08:00
Patrick Wendell
2fe39a4468
Some docs for the JavaTestUtils
2013-01-14 09:42:36 -08:00
Patrick Wendell
560c312c60
Docs, some tests, and work on
...
StreamingContext
2013-01-14 09:42:36 -08:00
Patrick Wendell
7e1049d8f1
Squashing a few TODOs
2013-01-14 09:42:36 -08:00
Patrick Wendell
74182010a4
Style cleanup and moving functions
2013-01-14 09:42:36 -08:00
Patrick Wendell
056f5efc55
More pair functions
2013-01-14 09:42:36 -08:00
Patrick Wendell
6e514a8d35
PairDStream and DStreamLike
2013-01-14 09:42:36 -08:00
Patrick Wendell
f144e0413a
Adding transform and union
2013-01-14 09:42:36 -08:00
Patrick Wendell
0d0bab25bd
Reduce tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
22a8c7be9a
Adding more tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
91b3d41448
Better equality test (thanks Josh)
2013-01-14 09:42:36 -08:00
Patrick Wendell
b0974e6c1d
Remving main method from tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
867a7455e2
Adding some initial tests to streaming API.
2013-01-14 09:42:36 -08:00
Patrick Wendell
b607c9e916
A very rough, early cut at some Java functionality for Streaming.
2013-01-14 09:42:36 -08:00
Tathagata Das
131be5d62e
Fixed bug in RDD checkpointing.
2013-01-14 03:28:25 -08:00
folone
25c0739bad
Moved to scala 2.10.0. Notable changes are:
...
- akka 2.0.3 → 2.1.0
- spray 1.0-M1 → 1.1-M7
For now the repl subproject is commented out, as scala reflection api changed very much since the introduction of macros.
2013-01-14 09:52:11 +01:00
Tathagata Das
82b0cc90ca
Merge pull request #370 from tdas/streaming
...
Added more documentation and minor change in API for NetworkReceiver
2013-01-13 21:28:12 -08:00
Tathagata Das
f90f794cde
Minor name fix
2013-01-13 21:25:57 -08:00
Tathagata Das
6cc8592f26
Fixed bug
2013-01-13 21:20:49 -08:00
Tathagata Das
0dbd411a56
Added documentation for PairDStreamFunctions.
2013-01-13 21:08:35 -08:00
Matei Zaharia
cb867e9ffb
Merge branch 'master' of github.com:mesos/spark
2013-01-13 19:34:32 -08:00
Matei Zaharia
72408e8dfa
Make filter preserve partitioner info, since it can
2013-01-13 19:34:07 -08:00
Tathagata Das
0a2e333341
Removed stream id from the constructor of NetworkReceiver to make it easier for PluggableNetworkInputDStream.
2013-01-13 16:18:39 -08:00
Matei Zaharia
9a34409810
Merge pull request #360 from rxin/cogroup-java
...
Changed CoGroupRDD's hash map from Scala to Java.
2013-01-13 15:31:08 -08:00
Reynold Xin
be7166146b
Removed the use of getOrElse to avoid Scala wrapper for every call.
2013-01-13 15:27:28 -08:00
Tathagata Das
23b54f62b7
Merge pull request #366 from tdas/streaming
...
Code refactoring
2013-01-13 14:07:34 -08:00
Matei Zaharia
530493b0e8
Merge pull request #365 from ryanlecompte/rate_limiter_timing_cleanup
...
Improve sleeping algorithm for rate limiting output streams
2013-01-13 10:47:00 -08:00
Ryan LeCompte
c31931af7e
switch to uppercase constants
2013-01-13 10:39:47 -08:00
Ryan LeCompte
2305a2c1d9
more code cleanup
2013-01-13 10:01:56 -08:00
Matei Zaharia
62e4767394
Merge pull request #368 from mbautin/add_spray_json_dependency
...
Add missing dependency spray-json to Maven build
2013-01-13 00:48:21 -08:00
Mikhail Bautin
88d8f11365
Add missing dependency spray-json to Maven build
2013-01-13 00:46:25 -08:00
Matei Zaharia
fbb3fc4143
Merge pull request #346 from JoshRosen/python-api
...
Python API (PySpark)
2013-01-12 23:49:36 -08:00
Matei Zaharia
44b3e41f2e
Merge pull request #367 from ericzhang-cn/master
...
fix spelling mistake in LocalLR example
2013-01-12 23:37:17 -08:00
Eric Zhang
ba06e9c97c
Update examples/src/main/scala/spark/examples/LocalLR.scala
...
fix spelling mistake
2013-01-13 15:33:11 +08:00
Matei Zaharia
01413ca0e7
Merge pull request #364 from tysonjh/master
...
Executor and JobDescription JSON support added
2013-01-12 16:17:07 -08:00
Matei Zaharia
3ef82496f4
Merge pull request #363 from heuermh/quick-start-fix
...
add repositories section to quick-start SimpleJob pom.xml
2013-01-12 16:15:10 -08:00
Matei Zaharia
995075bf79
Merge pull request #355 from shivaram/default-hadoop-pom
...
Activate hadoop1 profile by default for maven builds
2013-01-12 15:38:36 -08:00
Shivaram Venkataraman
bbc56d85ed
Rename environment variable for hadoop profiles to hadoopVersion
2013-01-12 15:24:13 -08:00
Ryan LeCompte
addff2c466
add comment
2013-01-12 09:57:29 -08:00
Ryan LeCompte
ea20ae6618
add one extra test
2013-01-12 09:18:00 -08:00
Ryan LeCompte
2c77eeebb6
correct test params
2013-01-12 00:13:45 -08:00
Ryan LeCompte
0cfea7a2ec
add unit test
2013-01-11 23:48:07 -08:00
Ryan LeCompte
ff10b3aa09
add missing return
2013-01-11 21:03:57 -08:00
Ryan LeCompte
22445fbea9
attempt to sleep for more accurate time period, minor cleanup
2013-01-11 13:30:49 -08:00
Tyson
1731f1fed4
Added an optional format parameter for individual job queries and optimized the jobId query
2013-01-11 15:01:43 -05:00
Tyson
c063e8777e
Added implicit json writers for JobDescription and ExecutorRunner
2013-01-11 14:57:38 -05:00