Matei Zaharia
|
a13780670d
|
Added a unit test for local-cluster mode and simplified some of the code involved in that
|
2012-09-10 12:48:58 -07:00 |
|
Denny
|
f2ac55840c
|
Add shutdown hook to Executor Runner and execute code to shutdown local cluster in Scheduler Backend
|
2012-09-10 12:48:58 -07:00 |
|
|
Denny
|
9ead8ab14e
|
Set SPARK_LAUNCH_WITH_SCALA=0 in Executor Runner
|
2012-09-10 12:48:58 -07:00 |
|
|
Denny
|
8bb3c73977
|
Renamed spark-cluster to spark-local.
|
2012-09-10 12:48:58 -07:00 |
|
|
Denny
|
a367c20f49
|
Fix wrong counting
|
2012-09-10 12:48:57 -07:00 |
|
|
Denny
|
93fe331e6d
|
Delete old DeployUtils.
|
2012-09-10 12:48:57 -07:00 |
|
|
Denny
|
cf074f9c96
|
Renamed class.
|
2012-09-10 12:48:57 -07:00 |
|
|
Denny
|
3749f94184
|
Start a standalone cluster locally.
|
2012-09-10 12:48:57 -07:00 |
|
Mosharaf Chowdhury
|
d0754e62c1
|
Merge remote-tracking branch 'upstream/dev' into dev
|
2012-09-08 08:54:38 -07:00 |
|
|
Matei Zaharia
|
995982b3c9
|
Added a unit test for local-cluster mode and simplified some of the code involved in that
|
2012-09-07 17:08:36 -07:00 |
|
|
Matei Zaharia
|
8d2fcc2832
|
Merge pull request #189 from dennybritz/feature/localcluster
Simulating a Spark standalone cluster locally
|
2012-09-07 15:43:43 -07:00 |
|
|
Denny
|
7ff9311add
|
Add shutdown hook to Executor Runner and execute code to shutdown local cluster in Scheduler Backend
|
2012-09-07 14:09:12 -07:00 |
|
Imran Rashid
|
f8f125eebb
|
fix stupid typo and add documentation
|
2012-09-07 13:58:46 -07:00 |
|
Tathagata Das
|
0269792c17
|
Merge branch 'dev' of github.com:radlab/spark into dev
Conflicts:
streaming/src/main/scala/spark/streaming/Scheduler.scala
|
2012-09-07 20:18:30 +00:00 |
|
|
Tathagata Das
|
b5750726ff
|
Fixed bugs in streaming Scheduler and optimized QueueInputDStream.
|
2012-09-07 20:16:21 +00:00 |
|
|
Denny
|
4e7b264cf7
|
Set SPARK_LAUNCH_WITH_SCALA=0 in Executor Runner
|
2012-09-07 11:39:44 -07:00 |
|
haoyuan
|
381e2c7ac4
|
add warmup code for TopKWordCountRaw.scala
|
2012-09-06 20:54:52 -07:00 |
|
|
haoyuan
|
0681bbc5d9
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-07 02:18:33 +00:00 |
|
|
haoyuan
|
db08a362aa
|
commit opt for grep scalibility test.
|
2012-09-07 02:17:52 +00:00 |
|
|
Tathagata Das
|
4a7bde6865
|
Fixed bugs and added testcases for naive reduceByKeyAndWindow.
|
2012-09-06 19:06:59 -07:00 |
|
root
|
c2da64409a
|
Randomize the order of block fetches in getMultiple
|
2012-09-06 23:16:26 +00:00 |
|
|
Tathagata Das
|
203ac8fa8b
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-06 05:29:06 -07:00 |
|
|
Tathagata Das
|
babb7e3ce2
|
Re-implemented ReducedWindowedDSteam to simplify and fix bugs. Added slice operator to DStream. Also, refactored DStream testsuites and added tests for reduceByKeyAndWindow.
|
2012-09-06 05:28:29 -07:00 |
|
|
Mosharaf Chowdhury
|
7c6936f1bc
|
Merge remote-tracking branch 'upstream/dev' into dev
|
2012-09-06 01:46:48 -07:00 |
|
root
|
019de4562c
|
Less warmup in word count
|
2012-09-06 02:50:41 +00:00 |
|
|
root
|
9ef90c95f4
|
Bug fix
|
2012-09-06 00:43:46 +00:00 |
|
|
root
|
2fa6d999fd
|
Tuning Akka more
|
2012-09-06 00:16:39 +00:00 |
|
|
Denny
|
886183e591
|
Renamed spark-cluster to spark-local.
|
2012-09-05 17:10:54 -07:00 |
|
|
root
|
215544820f
|
Serialize map output locations more efficiently, and only once, in MapOutputTracker
|
2012-09-05 23:54:04 +00:00 |
|
|
Matei Zaharia
|
53a5681c8a
|
Merge pull request #190 from rxin/dev
Log cache add/remove messages in block manager.
|
2012-09-05 16:41:52 -07:00 |
|
|
root
|
dc68febdce
|
User Spark's closure serializer for the ShuffleMapTask cache
|
2012-09-05 23:06:59 +00:00 |
|
Reynold Xin
|
c308fbcb79
|
Removed cache add/remove log messages from CacheTracker.
Added log messages on BlockManagerMaster to reflect block add/remove.
Also did some minor cleanup of storage package code.
|
2012-09-05 15:59:48 -07:00 |
|
|
root
|
ed937a821f
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-05 22:26:49 +00:00 |
|
|
root
|
1d6b36d3c3
|
Further tuning for network performance
|
2012-09-05 22:26:37 +00:00 |
|
root
|
3fa0d7f0c9
|
Serialize BlockRDD more efficiently
|
2012-09-05 08:28:15 +00:00 |
|
root
|
4a5d0d249e
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-05 08:23:09 +00:00 |
|
|
root
|
efc7668d16
|
Allow serializing HttpBroadcast through Kryo
|
2012-09-05 08:22:57 +00:00 |
|
|
root
|
75487b2f5a
|
Broadcast the JobConf in HadoopRDD to reduce task sizes
|
2012-09-05 08:14:50 +00:00 |
|
|
root
|
b7ad291ac5
|
Tuning Akka for more connections
|
2012-09-05 07:08:07 +00:00 |
|
|
root
|
fc186dc18a
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-05 05:53:18 +00:00 |
|
|
root
|
4ea032a142
|
Some changes to make important log output visible even if we set the logging to WARNING
|
2012-09-05 05:53:07 +00:00 |
|
|
Denny
|
babbca0a2f
|
Fix wrong counting
|
2012-09-04 22:04:18 -07:00 |
|
|
Denny
|
9326509f66
|
Delete old DeployUtils.
|
2012-09-04 21:15:23 -07:00 |
|
|
Denny
|
1588d4dbe6
|
Renamed class.
|
2012-09-04 21:13:25 -07:00 |
|
|
Denny
|
22dde6e020
|
Start a standalone cluster locally.
|
2012-09-04 20:56:30 -07:00 |
|
|
Tathagata Das
|
25fd684b89
|
Merge branch 'dev' of github.com:radlab/spark into dev
|
2012-09-04 20:44:14 -07:00 |
|
|
Tathagata Das
|
7c09ad0e04
|
Changed DStream member access permissions from private to protected. Updated StateDStream to checkpoint RDDs and forget lineage.
|
2012-09-04 19:11:49 -07:00 |
|
|
haoyuan
|
96a1f2277d
|
fix the compile error in TopKWordCountRaw.scala
|
2012-09-04 18:03:34 -07:00 |
|
|
haoyuan
|
2ff72f60ac
|
add TopKWordCountRaw.scala
|
2012-09-04 17:55:55 -07:00 |
|
|
Tathagata Das
|
389a78722c
|
Updated the return types of PairDStreamFunctions to return DStreams instead of ShuffleDStreams for cleaner abstraction.
|
2012-09-04 15:37:46 -07:00 |
|