Tathagata Das
a805ac4a7c
Disabled checkpoint for PairwiseRDD (pySpark).
2013-01-16 10:55:26 -08:00
seanm
a4639400ea
Merge branch 'streaming' into sm-updateStateByKey
2013-01-15 20:29:01 -07:00
Matei Zaharia
4beb084f64
Merge pull request #374 from woggling/null-mapout
...
Generate FetchFailedException even for cached missing map outputs
2013-01-15 14:22:29 -08:00
Matei Zaharia
7adfedb0d7
Merge pull request #378 from apsaltis/master
...
Updated SCALA_VERSION in run2.cmd to match runtime version of Scala
2013-01-15 14:20:44 -08:00
Tathagata Das
cd1521cfdb
Merge branch 'master' into streaming
...
Conflicts:
core/src/main/scala/spark/rdd/CoGroupedRDD.scala
core/src/main/scala/spark/rdd/FilteredRDD.scala
docs/_layouts/global.html
docs/index.md
run
2013-01-15 12:08:51 -08:00
Charles Reiss
4078623b9f
Remove broken attempt to test fetching case.
2013-01-15 12:05:54 -08:00
Stephen Haberman
74d3b23929
Add spark.executor.memory to differentiate executor memory from spark-shell memory.
2013-01-15 14:03:28 -06:00
Andrew Psaltis
c7143e5507
Changed teh scala version to 2.9.2, so that the classes can be found when the classpath is expanded.
2013-01-15 12:45:42 -07:00
Matei Zaharia
20adf27480
Merge pull request #371 from folone/master
...
Scala 2.10.0
2013-01-15 11:03:49 -08:00
Matei Zaharia
3a98611d6b
Merge pull request #377 from stephenh/executeOnCompleteCallbackInFinally
...
Call executeOnCompleteCallbacks in a finally block.
2013-01-15 10:38:04 -08:00
Stephen Haberman
d228bff440
Add a test.
2013-01-15 11:48:50 -06:00
Stephen Haberman
dd583b7ebf
Call executeOnCompleteCallbacks in a finally block.
2013-01-15 10:52:06 -06:00
Tathagata Das
eded21925a
Merge pull request #375 from tdas/streaming
...
Important bug fixes
2013-01-14 23:06:40 -08:00
Nick Pentreath
b77f7390a5
Python ALS example
2013-01-15 09:04:32 +02:00
Tathagata Das
1638fcb0dc
Fixed updateStateByKey to work with primitive types.
2013-01-14 17:18:39 -08:00
Charles Reiss
b038999797
Fix accidental spark.master.host reuse
2013-01-14 17:04:44 -08:00
seanm
c203a29296
StateDStream changes to give updateStateByKey consistent behavior
2013-01-14 17:22:03 -07:00
seanm
b61a4ec773
Removing offset management code that is non-existent in kafka 0.7.0+
2013-01-14 17:13:10 -07:00
Charles Reiss
7ba34bc007
Additional tests for MapOutputTracker.
2013-01-14 15:27:02 -08:00
Charles Reiss
273fb5cc10
Throw FetchFailedException for cached missing locs
2013-01-14 15:26:48 -08:00
Patrick Wendell
a0013beb03
Stash
2013-01-14 15:15:02 -08:00
Patrick Wendell
8ad6220bd3
Bugfix
2013-01-14 14:56:36 -08:00
Patrick Wendell
38d9a3a863
Remove AnyRef constraint in updateState
2013-01-14 13:31:49 -08:00
Patrick Wendell
ae5290f4a2
Bug fix
2013-01-14 13:31:32 -08:00
Patrick Wendell
6069446356
Making comments consistent w/ Spark style
2013-01-14 10:42:05 -08:00
Patrick Wendell
d182a57cae
Two changes:
...
- Updating countByX() types based on bug fix
- Porting new documentation to Java
2013-01-14 10:03:55 -08:00
Patrick Wendell
a292ed8d8a
Some style cleanup
2013-01-14 09:42:36 -08:00
Patrick Wendell
3461cd99b7
Flume example and bug fix
2013-01-14 09:42:36 -08:00
Patrick Wendell
5bcb048167
More work on InputStreams
2013-01-14 09:42:36 -08:00
Patrick Wendell
280b6d0186
Porting to new Duration class
2013-01-14 09:42:36 -08:00
Patrick Wendell
c2537057f9
Fixing issue with <Long> types
2013-01-14 09:42:36 -08:00
Patrick Wendell
b36c4f7cce
More work on StreamingContext
2013-01-14 09:42:36 -08:00
Patrick Wendell
5004eec37c
Import Cleanup
2013-01-14 09:42:36 -08:00
Patrick Wendell
2fe39a4468
Some docs for the JavaTestUtils
2013-01-14 09:42:36 -08:00
Patrick Wendell
560c312c60
Docs, some tests, and work on
...
StreamingContext
2013-01-14 09:42:36 -08:00
Patrick Wendell
7e1049d8f1
Squashing a few TODOs
2013-01-14 09:42:36 -08:00
Patrick Wendell
74182010a4
Style cleanup and moving functions
2013-01-14 09:42:36 -08:00
Patrick Wendell
056f5efc55
More pair functions
2013-01-14 09:42:36 -08:00
Patrick Wendell
6e514a8d35
PairDStream and DStreamLike
2013-01-14 09:42:36 -08:00
Patrick Wendell
f144e0413a
Adding transform and union
2013-01-14 09:42:36 -08:00
Patrick Wendell
0d0bab25bd
Reduce tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
22a8c7be9a
Adding more tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
91b3d41448
Better equality test (thanks Josh)
2013-01-14 09:42:36 -08:00
Patrick Wendell
b0974e6c1d
Remving main method from tests
2013-01-14 09:42:36 -08:00
Patrick Wendell
867a7455e2
Adding some initial tests to streaming API.
2013-01-14 09:42:36 -08:00
Patrick Wendell
b607c9e916
A very rough, early cut at some Java functionality for Streaming.
2013-01-14 09:42:36 -08:00
Tathagata Das
131be5d62e
Fixed bug in RDD checkpointing.
2013-01-14 03:28:25 -08:00
folone
25c0739bad
Moved to scala 2.10.0. Notable changes are:
...
- akka 2.0.3 → 2.1.0
- spray 1.0-M1 → 1.1-M7
For now the repl subproject is commented out, as scala reflection api changed very much since the introduction of macros.
2013-01-14 09:52:11 +01:00
Tathagata Das
82b0cc90ca
Merge pull request #370 from tdas/streaming
...
Added more documentation and minor change in API for NetworkReceiver
2013-01-13 21:28:12 -08:00
Tathagata Das
f90f794cde
Minor name fix
2013-01-13 21:25:57 -08:00