Holden Karau
d86dc74d79
Code review feedback
2014-01-05 22:05:30 -08:00
Patrick Wendell
604fad9c39
Merge remote-tracking branch 'apache-github/master' into remove-binaries
...
Conflicts:
core/src/test/scala/org/apache/spark/DriverSuite.scala
docs/python-programming-guide.md
2014-01-03 21:29:33 -08:00
Patrick Wendell
4ae101ff38
Merge pull request #317 from ScrapCodes/spark-915-segregate-scripts
...
Spark-915 segregate scripts
2014-01-03 11:24:35 -08:00
Prashant Sharma
74ba97fcf7
sbin/spark-class* -> bin/spark-class*
2014-01-03 15:08:01 +05:30
Prashant Sharma
94f2fffa23
fixed review comments
2014-01-03 14:43:37 +05:30
Raymond Liu
f442afc22e
fix docs for yarn
2014-01-03 14:14:35 +08:00
Raymond Liu
7815a3ace9
Update maven build documentation
2014-01-03 12:12:38 +08:00
Raymond Liu
be343d2a56
Fix yarn/README.md and update docs/running-on-yarn.md
2014-01-03 12:12:38 +08:00
Prashant Sharma
94b7a7fe37
run-example -> bin/run-example
2014-01-02 18:41:21 +05:30
Prashant Sharma
b810a85cdd
spark-shell -> bin/spark-shell
2014-01-02 18:37:40 +05:30
Prashant Sharma
980afd280a
Merge branch 'scripts-reorg' of github.com:shane-huang/incubator-spark into spark-915-segregate-scripts
...
Conflicts:
bin/spark-shell
core/pom.xml
core/src/main/scala/org/apache/spark/SparkContext.scala
core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/CoarseMesosSchedulerBackend.scala
core/src/main/scala/org/apache/spark/ui/UIWorkloadGenerator.scala
core/src/test/scala/org/apache/spark/DriverSuite.scala
python/run-tests
sbin/compute-classpath.sh
sbin/spark-class
sbin/stop-slaves.sh
2014-01-02 17:55:21 +05:30
Prashant Sharma
6be4c11194
Removed sbt folder and changed docs accordingly
2014-01-02 14:09:37 +05:30
Patrick Wendell
1291dd4dce
Fix list rendering in YARN markdown docs.
2013-12-10 16:38:33 -08:00
Patrick Wendell
41c60b337a
Various broken links in documentation
2013-12-07 22:31:44 -08:00
Ali Ghodsi
e2c2914faa
more docs
2013-12-06 16:54:06 -08:00
Ali Ghodsi
f2fb4b4228
Updated documentation about the YARN v2.2 build process
2013-12-06 16:31:26 -08:00
Raymond Liu
ab3cefde53
Add YarnClientClusterScheduler and Backend.
...
With this scheduler, the user application is launched locally,
While the executor will be launched by YARN on remote nodes.
This enables spark-shell to run upon YARN.
2013-11-22 09:23:27 +08:00
tgravescs
4093e9393a
Impove Spark on Yarn Error handling
2013-11-19 12:44:00 -06:00
tgravescs
a35472e1dd
Allow spark on yarn to be run from HDFS. Allows the spark.jar, app.jar, and log4j.properties to be put into hdfs.
2013-11-04 16:16:28 -06:00
Matei Zaharia
8f11c36fe1
Merge remote-tracking branch 'tgravescs/sparkYarnDistCache'
...
Closes #11
Conflicts:
docs/running-on-yarn.md
yarn/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
2013-10-10 19:34:33 -07:00
tgravescs
0fff4ee852
Adding in the --addJars option to make SparkContext.addJar work on yarn and cleanup
...
the classpaths
2013-10-03 11:52:16 -05:00
tgravescs
bc3b20abdc
Allow users to set the application name for Spark on Yarn
2013-10-02 12:54:17 -05:00
Y.CORP.YAHOO.COM\tgraves
9d4246863a
Support distributed cache files and archives on spark on yarn and attempt to cleanup the staging directory on exit
2013-09-23 09:09:59 -05:00
shane-huang
dfbdc9ddb7
added spark-class and spark-executor to sbin
...
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
2013-09-23 11:28:58 +08:00
Jey Kottalam
35ed09f1d1
Clarify YARN example
2013-09-06 11:31:16 -07:00
Y.CORP.YAHOO.COM\tgraves
c8cc276110
Review comment changes and update to org.apache packaging
2013-09-03 10:50:21 -05:00
Y.CORP.YAHOO.COM\tgraves
547fc4a412
Merge remote-tracking branch 'mesos/master' into yarnUILink
...
Conflicts:
core/src/main/scala/org/apache/spark/ui/UIUtils.scala
core/src/main/scala/org/apache/spark/ui/jobs/PoolTable.scala
core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala
docs/running-on-yarn.md
2013-09-03 08:36:59 -05:00
Y.CORP.YAHOO.COM\tgraves
96452eea56
fix up minor things
2013-08-30 16:04:31 -05:00
Y.CORP.YAHOO.COM\tgraves
bac46266a9
Link the Spark UI to the Yarn UI
2013-08-30 15:55:32 -05:00
Matei Zaharia
f3a964848d
More doc improvements + better warnings when you haven't built Spark
2013-08-30 12:41:25 -07:00
Matei Zaharia
53cd50c069
Change build and run instructions to use assemblies
...
This commit makes Spark invocation saner by using an assembly JAR to
find all of Spark's dependencies instead of adding all the JARs in
lib_managed. It also packages the examples into an assembly and uses
that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script
with two better-named scripts: "run-examples" for examples, and
"spark-class" for Spark internal classes (e.g. REPL, master, etc). This
is also designed to minimize the confusion people have in trying to use
"run" to run their own classes; it's not meant to do that, but now at
least if they look at it, they can modify run-examples to do a decent
job for them.
As part of this, Bagel's examples are also now properly moved to the
examples package instead of bagel.
2013-08-29 21:19:04 -07:00
Matei Zaharia
baa84e7e4c
Merge pull request #865 from tgravescs/fixtmpdir
...
Spark on Yarn should use yarn approved directories for spark.local.dir and tmp
2013-08-28 12:44:46 -07:00
Y.CORP.YAHOO.COM\tgraves
63dc635de6
fix typos
2013-08-26 17:06:20 -05:00
Y.CORP.YAHOO.COM\tgraves
c9464c74a1
Add ability for user to specify environment variables
2013-08-26 16:44:27 -05:00
Y.CORP.YAHOO.COM\tgraves
6dd64e8bb2
Update docs and remove old reference to --user option
2013-08-26 14:29:24 -05:00
Jey Kottalam
6585f49841
Update build docs
2013-08-21 14:51:56 -07:00
Jey Kottalam
14b6bcdf93
update YARN docs
2013-08-15 16:50:37 -07:00
Reynold Xin
0eab7a78b9
Fixed a couple typos and formating problems in the YARN documentation.
2013-05-17 18:05:46 -07:00
Mridul Muralidharan
da2642bead
Fix example jar name
2013-05-17 06:58:46 +05:30
Mridul Muralidharan
f16c781709
Fix documentation to use yarn-standalone as master
2013-05-16 17:50:22 +05:30
Mridul Muralidharan
87540a7b38
Fix running on yarn documentation
2013-05-16 15:27:58 +05:30
Mridul Muralidharan
ee37612bc9
1) Add support for HADOOP_CONF_DIR (and/or YARN_CONF_DIR - use either) : which is used to specify the client side configuration directory : which needs to be part of the CLASSPATH.
...
2) Move from var+=".." to var="$var.." : the former does not work on older bash shells unfortunately.
2013-05-11 11:12:22 +05:30
Mridul Muralidharan
ac2e8e8720
Add some basic documentation
2013-04-19 00:13:19 +05:30
Andy Konwinski
c9097628fc
Fix broken link to YARN documentation.
2013-03-13 14:51:13 -07:00
Patrick Wendell
8321e7f0c2
Fixing YARN instructions
2012-10-09 22:39:28 -07:00
Andy Konwinski
e1a724f39c
Updating lots of docs to use the new special version number variables,
...
also adding the version to the navbar so it is easy to tell which
version of Spark these docs were compiled for.
2012-10-08 17:17:17 -07:00
Matei Zaharia
bf18e0994e
Minor typos
2012-09-26 23:53:38 -07:00
Matei Zaharia
ea05fc130b
Updates to standalone cluster, web UI and deploy docs.
2012-09-26 22:54:39 -07:00
Matei Zaharia
d51d5e0582
Doc fixes
2012-09-25 23:59:04 -07:00
Matei Zaharia
f1246cc7c1
Various enhancements to the programming guide and HTML/CSS
2012-09-25 23:26:56 -07:00
Denny
6d53b971b9
Added standalone and YARN docs. Merged standalone cluster into standalone doc
2012-09-13 09:47:54 -07:00