ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2044 commits 6 branches 181 tags 397 MiB
Commit graph

196 commits

Author SHA1 Message Date
Matei Zaharia b29599e5cf Fix code that depended on metadata cleaner interval being in minutes 2013-01-28 22:24:47 -08:00
Josh Rosen 551a47a620 Refactor daemon thread pool creation. 2013-01-21 23:31:00 -08:00
Stephen Haberman e5ca241335 Move JavaAPISuite into spark.streaming. 2013-01-21 16:06:58 -06:00
Matei Zaharia 6e3754bf47 Add Maven build file for streaming, and fix some issues in SBT file 
As part of this, changed our Scala 2.9.2 Kafka library to be available
as a local Maven repository, following the example in
(http://blog.dub.podval.org/2010/01/maven-in-project-repository.html)
 2013-01-20 19:22:24 -08:00
Tathagata Das 33bad85bb9 Fixed streaming testsuite bugs 2013-01-20 03:51:11 -08:00
Tathagata Das 4f8fe58b25 Merge branch 'mesos-streaming' into streaming 
Conflicts:
	core/src/main/scala/spark/api/java/JavaRDDLike.scala
	core/src/main/scala/spark/api/java/JavaSparkContext.scala
	core/src/test/scala/spark/JavaAPISuite.java
 2013-01-20 01:13:56 -08:00
Tathagata Das 214345ceac Fixed issue https://spark-project.atlassian.net/browse/STREAMING-29, along with updates to doc comments in SparkContext.checkpoint(). 2013-01-19 23:50:17 -08:00
Patrick Wendell c46dd2de78 Moving tests to appropriate directory 2013-01-17 21:43:17 -08:00
Patrick Wendell e0165bf714 Adding queueStream and some slight refactoring 2013-01-17 21:25:49 -08:00
Patrick Wendell ee0314c3b3 Merge branch 'streaming' into streaming-java-api 2013-01-17 18:43:00 -08:00
Patrick Wendell 70ba994d6d Import fixup 2013-01-17 18:41:59 -08:00
Patrick Wendell 2261e62ee5 Style cleanup 2013-01-17 18:41:59 -08:00
Patrick Wendell 82b8707c6b Checkpointing in Streaming java API 2013-01-17 18:41:58 -08:00
Patrick Wendell 61b877c688 Adding flatMap 2013-01-17 18:41:58 -08:00
Patrick Wendell 8e6cbbc6c7 Adding other updateState functions 2013-01-17 18:41:58 -08:00
Patrick Wendell 2a872335c5 Bug fix and test cleanup 2013-01-17 18:41:58 -08:00
Tathagata Das 1638fcb0dc Fixed updateStateByKey to work with primitive types. 2013-01-14 17:18:39 -08:00
Patrick Wendell a0013beb03 Stash 2013-01-14 15:15:02 -08:00
Patrick Wendell 8ad6220bd3 Bugfix 2013-01-14 14:56:36 -08:00
Patrick Wendell 38d9a3a863 Remove AnyRef constraint in updateState 2013-01-14 13:31:49 -08:00
Patrick Wendell ae5290f4a2 Bug fix 2013-01-14 13:31:32 -08:00
Patrick Wendell 6069446356 Making comments consistent w/ Spark style 2013-01-14 10:42:05 -08:00
Patrick Wendell d182a57cae Two changes: 
- Updating countByX() types based on bug fix
- Porting new documentation to Java
 2013-01-14 10:03:55 -08:00
Patrick Wendell a292ed8d8a Some style cleanup 2013-01-14 09:42:36 -08:00
Patrick Wendell 3461cd99b7 Flume example and bug fix 2013-01-14 09:42:36 -08:00
Patrick Wendell 5bcb048167 More work on InputStreams 2013-01-14 09:42:36 -08:00
Patrick Wendell 280b6d0186 Porting to new Duration class 2013-01-14 09:42:36 -08:00
Patrick Wendell c2537057f9 Fixing issue with <Long> types 2013-01-14 09:42:36 -08:00
Patrick Wendell b36c4f7cce More work on StreamingContext 2013-01-14 09:42:36 -08:00
Patrick Wendell 5004eec37c Import Cleanup 2013-01-14 09:42:36 -08:00
Patrick Wendell 2fe39a4468 Some docs for the JavaTestUtils 2013-01-14 09:42:36 -08:00
Patrick Wendell 560c312c60 Docs, some tests, and work on 
StreamingContext
 2013-01-14 09:42:36 -08:00
Patrick Wendell 7e1049d8f1 Squashing a few TODOs 2013-01-14 09:42:36 -08:00
Patrick Wendell 74182010a4 Style cleanup and moving functions 2013-01-14 09:42:36 -08:00
Patrick Wendell 056f5efc55 More pair functions 2013-01-14 09:42:36 -08:00
Patrick Wendell 6e514a8d35 PairDStream and DStreamLike 2013-01-14 09:42:36 -08:00
Patrick Wendell f144e0413a Adding transform and union 2013-01-14 09:42:36 -08:00
Patrick Wendell 0d0bab25bd Reduce tests 2013-01-14 09:42:36 -08:00
Patrick Wendell 22a8c7be9a Adding more tests 2013-01-14 09:42:36 -08:00
Patrick Wendell 91b3d41448 Better equality test (thanks Josh) 2013-01-14 09:42:36 -08:00
Patrick Wendell b0974e6c1d Remving main method from tests 2013-01-14 09:42:36 -08:00
Patrick Wendell 867a7455e2 Adding some initial tests to streaming API. 2013-01-14 09:42:36 -08:00
Patrick Wendell b607c9e916 A very rough, early cut at some Java functionality for Streaming. 2013-01-14 09:42:36 -08:00
Tathagata Das f90f794cde Minor name fix 2013-01-13 21:25:57 -08:00
Tathagata Das 6cc8592f26 Fixed bug 2013-01-13 21:20:49 -08:00
Tathagata Das 0dbd411a56 Added documentation for PairDStreamFunctions. 2013-01-13 21:08:35 -08:00
Tathagata Das 0a2e333341 Removed stream id from the constructor of NetworkReceiver to make it easier for PluggableNetworkInputDStream. 2013-01-13 16:18:39 -08:00
Tathagata Das 365506fb03 Changed variable name form ***Time to ***Duration to keep things consistent. 2013-01-09 14:29:25 -08:00
Tathagata Das 156e8b47ef Split Time to Time (absolute instant of time) and Duration (duration of time). 2013-01-09 12:42:10 -08:00
Tathagata Das 8c1b872512 Moved Twitter example to the where the other examples are. 2013-01-07 17:48:10 -08:00