Matei Zaharia
d5bdd12353
Merge pull request #197 from andyk/doc
...
Adds a "docs" directory containing existing Spark documentation and doc build instructions
2012-09-12 13:50:50 -07:00
Andy Konwinski
49e98500a9
Updated base README to point to documentation site instead of wiki, updated
...
docs/README.md to describe use of Jekyll, and renmaed things to make them
more consistent with the lower-case-with-hyphens convention.
2012-09-12 13:03:43 -07:00
Andy Konwinski
16da942d66
Adding docs directory containing documentation currently on the wiki
...
which can be compiled via jekyll, using the command `jekyll`. To compile
and run a local webserver to serve the doc as a website, run
`jekyll --server`.
2012-09-12 13:03:43 -07:00
Matei Zaharia
a29ac5f9cf
Merge pull request #195 from dennybritz/feature/fileserver
...
Spark HTTP FileServer
2012-09-11 17:04:17 -07:00
Denny
5e4076e3f2
Merge branch 'dev' into feature/fileserver
...
Conflicts:
core/src/main/scala/spark/SparkContext.scala
2012-09-11 16:57:17 -07:00
Denny
77873d2c8e
Formatting
2012-09-11 16:51:46 -07:00
Denny
24b9b37314
Subclass URLClassLoader instead of using reflection
2012-09-11 16:51:08 -07:00
Denny
31c53e917d
Use stageId as index for fileSet caches.
2012-09-11 16:10:45 -07:00
Matei Zaharia
943df48348
Merge branch 'dev' of github.com:mesos/spark into dev
2012-09-11 16:00:37 -07:00
Matei Zaharia
6d7f907e73
Manually merge pull request #175 by Imran Rashid
2012-09-11 16:00:06 -07:00
Matei Zaharia
2498f95199
Fixed GrowableAccumulatorParam.zero() to do a copy and to make sure it's
...
empty instead of using the old initialValue
2012-09-11 15:17:44 -07:00
Matei Zaharia
4767ba5b7d
Merge pull request #196 from rxin/dev
...
Log entire exception (including stack trace) in BlockManagerWorker.
2012-09-11 14:40:44 -07:00
Reynold Xin
7af7c79ce5
Updated the logError call from the previous commit to conform to
...
logError API.
2012-09-11 14:32:24 -07:00
Matei Zaharia
66e5362ce0
Merge pull request #175 from squito/collection_accumulators
...
add accumulators for mutable collections, with correct typing!
2012-09-11 14:29:10 -07:00
Reynold Xin
38b9119c96
Log entire exception (including stack trace) in BlockManagerWorker.
2012-09-11 11:31:35 -07:00
Imran Rashid
1490d09b5d
Merge branch 'master' into collection_accumulators
...
Conflicts:
core/src/test/scala/spark/AccumulatorSuite.scala
2012-09-11 11:20:56 -07:00
Denny
4d3471dd07
Fix serialization bugs and added local cluster tests
2012-09-10 15:39:58 -07:00
Denny
94a7e82ba1
Merge branch 'dev' of git://github.com/mesos/spark into feature/fileserver
2012-09-10 12:54:20 -07:00
Denny
b864c36a30
Dynamically adding jar files and caching fileSets.
2012-09-10 12:49:09 -07:00
Denny
f275fb07da
General FileServer
...
A general fileserver for both JARs and regular files.
2012-09-10 12:48:59 -07:00
Matei Zaharia
a13780670d
Added a unit test for local-cluster mode and simplified some of the code involved in that
2012-09-10 12:48:58 -07:00
Denny
f2ac55840c
Add shutdown hook to Executor Runner and execute code to shutdown local cluster in Scheduler Backend
2012-09-10 12:48:58 -07:00
Denny
9ead8ab14e
Set SPARK_LAUNCH_WITH_SCALA=0 in Executor Runner
2012-09-10 12:48:58 -07:00
Denny
8bb3c73977
Renamed spark-cluster to spark-local.
2012-09-10 12:48:58 -07:00
Denny
a367c20f49
Fix wrong counting
2012-09-10 12:48:57 -07:00
Denny
93fe331e6d
Delete old DeployUtils.
2012-09-10 12:48:57 -07:00
Denny
cf074f9c96
Renamed class.
2012-09-10 12:48:57 -07:00
Denny
3749f94184
Start a standalone cluster locally.
2012-09-10 12:48:57 -07:00
Mosharaf Chowdhury
d0754e62c1
Merge remote-tracking branch 'upstream/dev' into dev
2012-09-08 08:54:38 -07:00
Matei Zaharia
995982b3c9
Added a unit test for local-cluster mode and simplified some of the code involved in that
2012-09-07 17:08:36 -07:00
Matei Zaharia
8d2fcc2832
Merge pull request #189 from dennybritz/feature/localcluster
...
Simulating a Spark standalone cluster locally
2012-09-07 15:43:43 -07:00
Denny
7ff9311add
Add shutdown hook to Executor Runner and execute code to shutdown local cluster in Scheduler Backend
2012-09-07 14:09:12 -07:00
Imran Rashid
f8f125eebb
fix stupid typo and add documentation
2012-09-07 13:58:46 -07:00
Denny
4e7b264cf7
Set SPARK_LAUNCH_WITH_SCALA=0 in Executor Runner
2012-09-07 11:39:44 -07:00
root
c2da64409a
Randomize the order of block fetches in getMultiple
2012-09-06 23:16:26 +00:00
Mosharaf Chowdhury
7c6936f1bc
Merge remote-tracking branch 'upstream/dev' into dev
2012-09-06 01:46:48 -07:00
Denny
886183e591
Renamed spark-cluster to spark-local.
2012-09-05 17:10:54 -07:00
Matei Zaharia
53a5681c8a
Merge pull request #190 from rxin/dev
...
Log cache add/remove messages in block manager.
2012-09-05 16:41:52 -07:00
Reynold Xin
c308fbcb79
Removed cache add/remove log messages from CacheTracker.
...
Added log messages on BlockManagerMaster to reflect block add/remove.
Also did some minor cleanup of storage package code.
2012-09-05 15:59:48 -07:00
Denny
babbca0a2f
Fix wrong counting
2012-09-04 22:04:18 -07:00
Denny
9326509f66
Delete old DeployUtils.
2012-09-04 21:15:23 -07:00
Denny
1588d4dbe6
Renamed class.
2012-09-04 21:13:25 -07:00
Denny
22dde6e020
Start a standalone cluster locally.
2012-09-04 20:56:30 -07:00
Matei Zaharia
a842c63044
Minor formatting fixes
2012-09-03 16:24:00 -07:00
Mosharaf Chowdhury
5e8e8e4c9d
Merge remote-tracking branch 'upstream/dev' into dev
2012-09-03 16:16:56 -07:00
Matei Zaharia
2d6a629f8c
Merge pull request #182 from HarveyFeng/dev-fetch
...
Add a limit on the number of parallel fetches in the reduce stage
2012-09-03 16:14:57 -07:00
Harvey
3076b038f4
Start fetching a remote block when a received remote block has been passed
...
to the reduce function
2012-09-01 12:01:35 -07:00
Mosharaf Chowdhury
10dfee7501
Merge remote-tracking branch 'upstream/dev' into dev
2012-09-01 00:33:00 -07:00
Matei Zaharia
389fb4cc54
End runJob() with a SparkException when a task fails too many times in
...
one of the cluster schedulers.
2012-08-31 17:47:43 -07:00
Matei Zaharia
607b8fffcd
End runJob with a SparkException when a Mesos task fails too many times
2012-08-31 11:40:12 -07:00