ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3979 commits 6 branches 181 tags 397 MiB
Commit graph

3979 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia 651a96adf7 More fair scheduler docs and property names. 
Also changed uses of "job" terminology to "application" when they
referred to an entire Spark program, to avoid confusion.
 2013-09-08 00:29:11 -07:00
Matei Zaharia 98fb69822c Work in progress: 
- Add job scheduling docs
- Rename some fair scheduler properties
- Organize intro page better
- Link to Apache wiki for "contributing to Spark"
 2013-09-08 00:29:11 -07:00
Matei Zaharia 38488aca8a Merge pull request #900 from pwendell/cdh-docs 
Provide docs to describe running on CDH/HDP cluster.
 2013-09-08 00:28:53 -07:00
Patrick Wendell a8e376ec0f Merge pull request #904 from pwendell/master 
Adding Apache license to two files
 2013-09-07 21:16:01 -07:00
Patrick Wendell 6d2198643c Adding Apache license to two files 2013-09-07 20:46:58 -07:00
Patrick Wendell 22b982d2bc File rename 2013-09-07 14:38:54 -07:00
Matei Zaharia cfde85e395 Merge pull request #901 from ooyala/2013-09/0.8-doc-changes 
0.8 Doc changes for make-distribution.sh
 2013-09-07 13:53:08 -07:00
Matei Zaharia 4a7813a247 Merge pull request #903 from rxin/resulttask 
Fixed the bug that ResultTask was not properly deserializing outputId.
 2013-09-07 13:52:24 -07:00
Patrick Wendell 61c4762d45 Changes based on feedback 2013-09-07 11:55:10 -07:00
Evan Chan be1ee28ca6 CR feedback from Matei 2013-09-07 08:56:24 -07:00
Matei Zaharia afe46ba36e Merge pull request #892 from jey/fix-yarn-assembly 
YARN build fixes
 2013-09-07 07:28:51 -07:00
Reynold Xin 210eae26f4 Fixed the bug that ResultTask was not properly deserializing outputId. 2013-09-07 21:59:47 +08:00
Patrick Wendell 2eebeff5eb Merge pull request #897 from pwendell/master 
Docs describing Spark monitoring and instrumentation
 2013-09-06 15:25:22 -07:00
Evan Chan ff1dbf2106 Add references to make-distribution.sh 2013-09-06 14:20:44 -07:00
Evan Chan 88d53f0dff "launch" scripts is more accurate terminology 2013-09-06 14:03:44 -07:00
Evan Chan 5a18b854a7 Easier way to start the master 2013-09-06 13:59:43 -07:00
Evan Chan 76d5d2d3c5 Add notes about starting spark-shell 2013-09-06 13:53:00 -07:00
Patrick Wendell a2a0cf9d68 Docs describing Spark monitoring and instrumentation 2013-09-06 13:52:57 -07:00
Patrick Wendell e653a9d891 Provide docs to describe running on CDH/HDP cluster. 
This doc consolidates information relevant to CDH/HDP users in a single place.
 2013-09-06 13:49:57 -07:00
Jey Kottalam 30a32c8335 Minor YARN build cleanups 2013-09-06 11:31:16 -07:00
Jey Kottalam 70661246fd Fix YARN assembly generation under Maven 2013-09-06 11:31:16 -07:00
Jey Kottalam 35ed09f1d1 Clarify YARN example 2013-09-06 11:31:16 -07:00
Reynold Xin 1e15feb5a3 Hot fix to resolve the compilation error caused by SPARK-821. 2013-09-06 22:44:05 +08:00
Patrick Wendell ddcb9d310a Merge pull request #895 from ilikerps/821 
SPARK-821: Don't cache results when action run locally on driver
 2013-09-05 23:54:09 -07:00
Aaron Davidson 3a04e76c89 Reynold's second round of comments 2013-09-05 21:43:26 -07:00
Matei Zaharia 699c331f2f Merge pull request #891 from xiajunluan/SPARK-864 
[SPARK-864]DAGScheduler Exception if we delete Worker and StandaloneExecutorBackend then add Worker
 2013-09-05 20:21:53 -07:00
Aaron Davidson 4f2236a1c5 Add unit test and address comments 2013-09-05 18:06:30 -07:00
Aaron Davidson 1418d18af4 SPARK-821: Don't cache results when action run locally on driver 
Caching the results of local actions (e.g., rdd.first()) causes the driver to
store entire partitions in its own memory, which may be highly constrained.
This patch simply makes the CacheManager avoid caching the result of all locally-run computations.
 2013-09-05 15:34:42 -07:00
Andrew xia 7c15e3c5de Fix bug SPARK-864 2013-09-05 15:56:11 +08:00
Patrick Wendell 5c7494d7c1 Merge pull request #893 from ilikerps/master 
SPARK-884: Add unit test to validate Spark JSON output
 2013-09-04 22:47:03 -07:00
Aaron Davidson 714e7f9e32 Fix line over 100 chars 2013-09-04 22:40:08 -07:00
Aaron Davidson 37db141aef Address Patrick's comments 2013-09-04 21:34:20 -07:00
Matei Zaharia a54786678f Merge pull request #894 from c0s/master 
Updating assembly README to reflect recent changes in the build.
 2013-09-04 21:11:56 -07:00
Konstantin Boudnik 7c7c7e10ca Updating assembly README to reflect recent changes in the build. 2013-09-04 20:54:35 -07:00
Aaron Davidson 9e6f2b6822 SPARK-884: Add unit test to validate Spark JSON output 
This unit test simply validates that the outputs of
the JsonProtocol methods are syntactically valid JSON.
 2013-09-04 15:26:46 -07:00
Patrick Wendell c592a3c9b9 Minor spacing fix 2013-09-03 14:39:11 -07:00
Patrick Wendell 19f70273d2 Merge pull request #878 from tgravescs/yarnUILink 
Link the Spark UI up to the Yarn UI
 2013-09-03 14:29:10 -07:00
Matei Zaharia 68df2464d1 Merge pull request #889 from alig/master 
Return the port the WebUI is bound to (useful if port 0 was used)
 2013-09-03 13:01:17 -07:00
Y.CORP.YAHOO.COM\tgraves 41c1b5b9a0 Update based on review comments. Change function to prependBaseUri and fix formatting. 2013-09-03 14:46:51 -05:00
Y.CORP.YAHOO.COM\tgraves c8cc276110 Review comment changes and update to org.apache packaging 2013-09-03 10:50:21 -05:00
Y.CORP.YAHOO.COM\tgraves 547fc4a412 Merge remote-tracking branch 'mesos/master' into yarnUILink 
Conflicts:
	core/src/main/scala/org/apache/spark/ui/UIUtils.scala
	core/src/main/scala/org/apache/spark/ui/jobs/PoolTable.scala
	core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala
	docs/running-on-yarn.md
 2013-09-03 08:36:59 -05:00
Ali Ghodsi b25918d841 Merge branch 'master' of https://github.com/alig/spark 
Conflicts:
	core/src/main/scala/org/apache/spark/deploy/master/Master.scala
 2013-09-03 00:56:12 -07:00
Ali Ghodsi bd0788505f Using configured akka timeouts 2013-09-03 00:50:35 -07:00
Ali Ghodsi cbfef9b3ff Sort order of imports to match project guidelines 2013-09-02 19:33:55 -07:00
Ali Ghodsi 36d8fca2cc Reynold's comment fixed 2013-09-02 19:31:09 -07:00
Ali Ghodsi e452bd6d77 Brushing the code up slightly 2013-09-02 19:04:08 -07:00
Matei Zaharia 59218bdd49 Add Apache parent POM 2013-09-02 18:34:03 -07:00
Ali Ghodsi cf7b115496 Enabling getting the actual WEBUI port 2013-09-02 18:21:21 -07:00
Matei Zaharia d3dd48f8b3 Merge pull request #887 from mateiz/misc-fixes 
Miscellaneous fixes for 0.8
 2013-09-02 16:44:54 -07:00
Matei Zaharia f586c8ef38 Updated LICENSE with third-party licenses 2013-09-02 16:43:06 -07:00