ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
5198 commits 6 branches 181 tags 397 MiB
Commit graph

378 commits

Author SHA1 Message Date
Patrick Wendell c3cf0475e8 Merge pull request #339 from ScrapCodes/conf-improvements 
Conf improvements

There are two new features.

1. Allow users to set arbitrary akka configurations via spark conf.

2. Allow configuration to be printed in logs for diagnosis.
 2014-01-07 00:54:25 -08:00
Reynold Xin a862cafacf Merge pull request #331 from holdenk/master 
Add a script to download sbt if not present on the system

As per the discussion on the dev mailing list this script will use the system sbt if present or otherwise attempt to install the sbt launcher. The fall back error message in the event it fails instructs the user to install sbt. While the URLs it fetches from aren't controlled by the spark project directly, they are stable and the current authoritative sources.
 2014-01-07 00:18:20 -08:00
Prashant Sharma c729fa7c8e formatting related fixes suggested by Patrick. 2014-01-07 13:08:16 +05:30
Prashant Sharma b84dc780d3 Allow configuration to be printed in logs for diagnosis. 2014-01-07 13:01:43 +05:30
Prashant Sharma b3018811e1 Allow users to set arbitrary akka configurations via spark conf. 2014-01-07 13:01:43 +05:30
Andrew Ash 2dd4fb5698 Clarify spark.cores.max 
It controls the count of cores across the cluster, not on a per-machine basis.
 2014-01-06 09:01:46 -08:00
Holden Karau d86dc74d79 Code review feedback 2014-01-05 22:05:30 -08:00
Patrick Wendell 604fad9c39 Merge remote-tracking branch 'apache-github/master' into remove-binaries 
Conflicts:
	core/src/test/scala/org/apache/spark/DriverSuite.scala
	docs/python-programming-guide.md
 2014-01-03 21:29:33 -08:00
Patrick Wendell 4ae101ff38 Merge pull request #317 from ScrapCodes/spark-915-segregate-scripts 
Spark-915 segregate scripts
 2014-01-03 11:24:35 -08:00
Prashant Sharma 74ba97fcf7 sbin/spark-class* -> bin/spark-class* 2014-01-03 15:08:01 +05:30
Prashant Sharma 94f2fffa23 fixed review comments 2014-01-03 14:43:37 +05:30
Prashant Sharma b4bb80002b Merge branch 'master' into spark-1002-remove-jars 2014-01-03 12:12:04 +05:30
Raymond Liu f442afc22e fix docs for yarn 2014-01-03 14:14:35 +08:00
Raymond Liu ebdfa6bb97 Using name yarn-alpha/yarn instead of yarn-2.0/yarn-2.2 2014-01-03 12:14:38 +08:00
Raymond Liu 7815a3ace9 Update maven build documentation 2014-01-03 12:12:38 +08:00
Raymond Liu be343d2a56 Fix yarn/README.md and update docs/running-on-yarn.md 2014-01-03 12:12:38 +08:00
Prashant Sharma 59e8009b8d a few left over document change 2014-01-02 21:48:44 +05:30
Prashant Sharma a3f90a2ecf pyspark -> bin/pyspark 2014-01-02 18:50:12 +05:30
Prashant Sharma 94b7a7fe37 run-example -> bin/run-example 2014-01-02 18:41:21 +05:30
Prashant Sharma b810a85cdd spark-shell -> bin/spark-shell 2014-01-02 18:37:40 +05:30
Prashant Sharma 980afd280a Merge branch 'scripts-reorg' of github.com:shane-huang/incubator-spark into spark-915-segregate-scripts 
Conflicts:
	bin/spark-shell
	core/pom.xml
	core/src/main/scala/org/apache/spark/SparkContext.scala
	core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
	core/src/main/scala/org/apache/spark/ui/UIWorkloadGenerator.scala
	core/src/test/scala/org/apache/spark/DriverSuite.scala
	python/run-tests
	sbin/compute-classpath.sh
	sbin/spark-class
	sbin/stop-slaves.sh
 2014-01-02 17:55:21 +05:30
Prashant Sharma 6be4c11194 Removed sbt folder and changed docs accordingly 2014-01-02 14:09:37 +05:30
Matei Zaharia 0fa5809768 Updated docs for SparkConf and handled review comments 2013-12-30 22:17:28 -05:00
Reynold Xin 72a17b69f5 Revert "Merge pull request #310 from jyunfan/master" 
This reverts commit 79b20e4dbe, reversing
changes made to 7375047d51.
 2013-12-28 21:25:40 -10:00
Jyun-Fan Tsai 17f6620a71 Fix typo in the Accumulators section 
val => var
 2013-12-29 11:30:02 +08:00
fengdong ad8ce0148a changed the example links in the scala-programming-guid 2013-12-18 19:03:32 +08:00
fengdong ddebaf8280 Fixed the example link. 2013-12-18 11:00:36 +08:00
Reynold Xin 7db9165961 Merge pull request #251 from pwendell/master 
Fix list rendering in YARN markdown docs.

This is some minor clean-up which makes the list render correctly.
 2013-12-14 14:16:34 -08:00
Prashant Sharma d3090b79a5 A few corrections to documentation. 2013-12-12 10:12:06 +05:30
Prashant Sharma 603af51bb5 Merge branch 'master' into akka-bug-fix 
Conflicts:
	core/pom.xml
	core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala
	pom.xml
	project/SparkBuild.scala
	streaming/pom.xml
	yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
 2013-12-11 10:21:53 +05:30
Patrick Wendell 1291dd4dce Fix list rendering in YARN markdown docs. 2013-12-10 16:38:33 -08:00
Patrick Wendell 0428145ed4 Small fix 2013-12-07 22:33:11 -08:00
Patrick Wendell b3e87c0f51 Adding HDP 2.0 version 2013-12-07 22:31:46 -08:00
Patrick Wendell 41c60b337a Various broken links in documentation 2013-12-07 22:31:44 -08:00
Patrick Wendell 6494d62fe4 Merge pull request #240 from pwendell/master 
SPARK-917 Improve API links in nav bar
 2013-12-07 11:56:16 -08:00
Patrick Wendell dd331a6b26 SPARK-917 Improve API links in nav bar 2013-12-07 11:49:49 -08:00
Aaron Davidson cb6ac8aafb Correct spellling error in configuration.md 2013-12-07 01:40:01 -08:00
Patrick Wendell 7a1d1c93b8 Minor formatting fix in config file 2013-12-06 20:28:22 -08:00
Patrick Wendell 1b38f5f277 Merge pull request #236 from pwendell/shuffle-docs 
Adding disclaimer for shuffle file consolidation
 2013-12-06 20:16:15 -08:00
Patrick Wendell b9451acdf4 Adding disclaimer for shuffle file consolidation 2013-12-06 19:25:28 -08:00
Patrick Wendell bb6e25c663 Minor doc fixes and updating README 2013-12-06 17:42:28 -08:00
Ali Ghodsi e2c2914faa more docs 2013-12-06 16:54:06 -08:00
Ali Ghodsi f2fb4b4228 Updated documentation about the YARN v2.2 build process 2013-12-06 16:31:26 -08:00
Patrick Wendell 5d460253d6 Merge pull request #228 from pwendell/master 
Document missing configs and set shuffle consolidation to false.
 2013-12-05 12:31:24 -08:00
Patrick Wendell 1450b8ef87 Small changes from Matei review 2013-12-04 18:49:32 -08:00
Patrick Wendell b1c6fa1584 Document missing configs and set shuffle consolidation to false. 2013-12-04 18:39:34 -08:00
Andrew Ash 0c5af38b86 Typo: applicaton 2013-12-04 12:30:25 -08:00
Prashant Sharma 17987778da Merge branch 'master' into wip-scala-2.10 
Conflicts:
	core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala
	core/src/main/scala/org/apache/spark/rdd/MapPartitionsRDD.scala
	core/src/main/scala/org/apache/spark/rdd/MapPartitionsWithContextRDD.scala
	core/src/main/scala/org/apache/spark/rdd/RDD.scala
	python/pyspark/rdd.py
 2013-11-27 14:44:12 +05:30
Prashant Sharma 54862af5ee Improvements from the review comments and followed Boy Scout Rule. 2013-11-27 14:26:28 +05:30
Prashant Sharma dca946ff67 Documenting the newly added spark properties. 2013-11-26 20:47:38 +05:30