ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
5434 commits 6 branches 181 tags 397 MiB
Commit graph

35 commits

Author SHA1 Message Date
Patrick Wendell bc81ce040d Merge remote-tracking branch 'apache-github/master' into standalone-driver 
Conflicts:
	core/src/test/scala/org/apache/spark/deploy/JsonProtocolSuite.scala
	pom.xml
 2014-01-08 00:38:31 -08:00
Patrick Wendell c78b381e91 Fixes 2014-01-08 00:09:12 -08:00
Matei Zaharia d8bcc8e9a0 Add way to limit default # of cores used by applications on standalone mode 
Also documents the spark.deploy.spreadOut option.
 2014-01-07 14:35:52 -05:00
Patrick Wendell b72cceba27 Some doc fixes 2014-01-06 22:05:53 -08:00
Patrick Wendell c0498f9265 Merge remote-tracking branch 'apache-github/master' into standalone-driver 
Conflicts:
	core/src/main/scala/org/apache/spark/deploy/client/AppClient.scala
	core/src/main/scala/org/apache/spark/deploy/client/TestClient.scala
	core/src/main/scala/org/apache/spark/deploy/master/Master.scala
	core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala
	core/src/main/scala/org/apache/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala
 2014-01-06 17:29:21 -08:00
Prashant Sharma 74ba97fcf7 sbin/spark-class* -> bin/spark-class* 2014-01-03 15:08:01 +05:30
Prashant Sharma 59e8009b8d a few left over document change 2014-01-02 21:48:44 +05:30
Prashant Sharma b810a85cdd spark-shell -> bin/spark-shell 2014-01-02 18:37:40 +05:30
Prashant Sharma 980afd280a Merge branch 'scripts-reorg' of github.com:shane-huang/incubator-spark into spark-915-segregate-scripts 
Conflicts:
	bin/spark-shell
	core/pom.xml
	core/src/main/scala/org/apache/spark/SparkContext.scala
	core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
	core/src/main/scala/org/apache/spark/ui/UIWorkloadGenerator.scala
	core/src/test/scala/org/apache/spark/DriverSuite.scala
	python/run-tests
	sbin/compute-classpath.sh
	sbin/spark-class
	sbin/stop-slaves.sh
 2014-01-02 17:55:21 +05:30
Matei Zaharia 0fa5809768 Updated docs for SparkConf and handled review comments 2013-12-30 22:17:28 -05:00
Patrick Wendell 6ffa9bb226 Documentation and adding supervise option 2013-12-29 11:26:56 -08:00
Andrew Ash 0c5af38b86 Typo: applicaton 2013-12-04 12:30:25 -08:00
Aaron Davidson 66c20635fa Minor clarification and cleanup to spark-standalone.md 2013-10-10 14:45:12 -07:00
Aaron Davidson 42d8b8efe6 Address Matei's comments on documentation 
Updates to the documentation and changing some logError()s to logWarning()s.
 2013-10-10 00:33:47 -07:00
Aaron Davidson 4ea8ee468f Add docs for standalone scheduler fault tolerance 
Also fix a couple HTML/Markdown issues in other files.
 2013-10-08 14:18:31 -07:00
shane-huang fcfe4f9204 add admin scripts to sbin 
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
 2013-09-23 12:42:34 +08:00
shane-huang dfbdc9ddb7 added spark-class and spark-executor to sbin 
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
 2013-09-23 11:28:58 +08:00
Matei Zaharia 651a96adf7 More fair scheduler docs and property names. 
Also changed uses of "job" terminology to "application" when they
referred to an entire Spark program, to avoid confusion.
 2013-09-08 00:29:11 -07:00
Evan Chan be1ee28ca6 CR feedback from Matei 2013-09-07 08:56:24 -07:00
Evan Chan ff1dbf2106 Add references to make-distribution.sh 2013-09-06 14:20:44 -07:00
Evan Chan 88d53f0dff "launch" scripts is more accurate terminology 2013-09-06 14:03:44 -07:00
Evan Chan 5a18b854a7 Easier way to start the master 2013-09-06 13:59:43 -07:00
Evan Chan 76d5d2d3c5 Add notes about starting spark-shell 2013-09-06 13:53:00 -07:00
Matei Zaharia 3db404a43a Run script fixes for Windows after package & assembly change 2013-09-01 23:45:57 +00:00
Matei Zaharia 4819baa658 More updates, describing changes to recommended use of environment vars 
and new Python stuff
 2013-08-31 14:21:10 -07:00
Matei Zaharia 53cd50c069 Change build and run instructions to use assemblies 
This commit makes Spark invocation saner by using an assembly JAR to
find all of Spark's dependencies instead of adding all the JARs in
lib_managed. It also packages the examples into an assembly and uses
that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script
with two better-named scripts: "run-examples" for examples, and
"spark-class" for Spark internal classes (e.g. REPL, master, etc). This
is also designed to minimize the confusion people have in trying to use
"run" to run their own classes; it's not meant to do that, but now at
least if they look at it, they can modify run-examples to do a decent
job for them.

As part of this, Bagel's examples are also now properly moved to the
examples package instead of bagel.
 2013-08-29 21:19:04 -07:00
Andrew Ash afc2c80fdb Update spark-standalone.md 2013-08-07 00:44:43 +01:00
Matei Zaharia 05d2e94838 Use a separate memory setting for standalone cluster daemons 
Conflicts:
	docs/_config.yml
 2013-02-10 21:59:41 -08:00
Matei Zaharia 2173f6c7ca Clarify the documentation on env variables for standalone mode 2013-01-21 13:03:20 -08:00
Josh Rosen 1948f46093 Use spark-env.sh to configure standalone master. See SPARK-638. 
Also fixed a typo in the standalone mode documentation.
 2012-12-14 01:20:00 +00:00
Andy Konwinski 45d03231d0 Adds liquid variables to docs templating system so that they can be used 
throughout the docs: SPARK_VERSION, SCALA_VERSION, and MESOS_VERSION.

To use them, e.g. use {{site.SPARK_VERSION}}.

Also removes uses of {{HOME_PATH}} which were being resolved to ""
by the templating system anyway.
 2012-10-08 10:30:38 -07:00
Matei Zaharia ea05fc130b Updates to standalone cluster, web UI and deploy docs. 2012-09-26 22:54:39 -07:00
Matei Zaharia 56c90485fd More updates to documentation 2012-09-25 19:31:07 -07:00
Andy Konwinski 52c29071a4 - Add docs/api to .gitignore 
- Rework/expand the nav bar with more of the docs site
- Removing parts of docs about EC2 and Mesos that differentiate between
  running 0.5 and before
    - Merged subheadings from running-on-amazon-ec2.html that are still relevant
      (i.e., "Using a newer version of Spark" and "Accessing Data in S3") into
      ec2-scripts.html and deleted running-on-amazon-ec2.html
- Added some TODO comments to a few docs
- Updated the blurb about AMP Camp
- Renamed programming-guide to spark-programming-guide
- Fixing typos/etc. in Standalone Spark doc
 2012-09-16 15:28:52 -07:00
Denny 6d53b971b9 Added standalone and YARN docs. Merged standalone cluster into standalone doc 2012-09-13 09:47:54 -07:00