ctn
a1d2c34346
[BUGFIX] Fix for sbt/sbt script SPARK_HOME setting
...
In some environments, this command
export SPARK_HOME=$(cd "$(dirname $0)/.."; pwd)
echoes two paths, one by the "cd ..", and one by the "pwd". Note the resulting
erroneous -jar paths below:
ctn@ubuntu:~/src/spark$ sbt/sbt
+ EXTRA_ARGS=
+ '[' '' '!=' '' ']'
+++ dirname sbt/sbt
++ cd sbt/..
++ pwd
+ export 'SPARK_HOME=/home/ctn/src/spark
/home/ctn/src/spark'
+ SPARK_HOME='/home/ctn/src/spark
/home/ctn/src/spark'
+ export SPARK_TESTING=1
+ SPARK_TESTING=1
+ java -Xmx1200m -XX:MaxPermSize=350m -XX:ReservedCodeCacheSize=128m -jar /home/ctn/src/spark /home/ctn/src/spark/sbt/sbt-launch-0.11.3-2.jar
Error: Invalid or corrupt jarfile /home/ctn/src/spark
Committer: ctn <ctn@adatao.com>
On branch master
Changes to be committed:
- Send output of the "cd .." part to /dev/null
modified: sbt/sbt
2013-07-17 11:27:11 -07:00
ctn
b29c53e6aa
Merge remote-tracking branch 'upstream/master'
2013-07-17 11:24:03 -07:00
Matei Zaharia
cad48edb70
Merge pull request #708 from ScrapCodes/dependencies-upgrade
...
Dependency upgrade Akka 2.0.3 -> 2.0.5
2013-07-16 21:41:28 -07:00
Matei Zaharia
7e74ab5b0f
Merge remote-tracking branch 'origin/pr/704'
...
Conflicts:
make-distribution.sh
2013-07-16 19:13:07 -07:00
Matei Zaharia
4ff494de20
Some missing license headers
2013-07-16 17:26:48 -07:00
Matei Zaharia
af3c9d5042
Add Apache license headers and LICENSE and NOTICE files
2013-07-16 17:21:33 -07:00
Matei Zaharia
00a14deb6d
Update to latest Scala Maven plugin and allow Zinc external compiler
2013-07-16 11:52:20 -07:00
Matei Zaharia
b1f9f64743
Merge branch 'master' of github.com:mesos/spark
2013-07-16 11:01:53 -07:00
Matei Zaharia
5c388808a8
SPARK-814: Result stages should be named after action
2013-07-16 11:01:14 -07:00
Matei Zaharia
f347cc3f65
Fix deprecation warning and style issues
2013-07-16 10:53:30 -07:00
Prashant Sharma
2748e73eb9
Dependency upgrade Akka 2.0.3 -> 2.0.5
2013-07-16 16:08:46 +05:30
Matei Zaharia
8a8a8f2de2
Merge pull request #705 from rxin/errormessages
...
Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions.
2013-07-15 23:09:21 -07:00
Reynold Xin
69316603d6
Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions.
2013-07-15 22:50:11 -07:00
seanm
90b0142985
adding files generated from make-distribution.sh to .gitignore
2013-07-15 19:13:39 -06:00
seanm
a96b4ef761
dding tgz option to make-distribution.sh
2013-07-15 19:13:17 -06:00
Matei Zaharia
ed8415b2fa
Merge pull request #703 from karenfeng/ui-802
...
Link to job UI from standalone deploy cluster web UI
2013-07-15 16:41:04 -07:00
Karen Feng
6dc7c9bfb1
Removed job UI column, linked description to job UI
2013-07-15 16:33:50 -07:00
Karen Feng
fbf5aa761e
Removed log message, added field in master UI to link to log UI
2013-07-15 15:50:03 -07:00
Karen Feng
eac381a957
Merge branch 'ui-802' of https://github.com/karenfeng/spark into ui-802
2013-07-15 15:48:44 -07:00
Karen Feng
3955711250
Added field to master UI with link to job UI
2013-07-15 15:47:21 -07:00
Matei Zaharia
e3d3e6f0ab
Merge pull request #702 from karenfeng/ui-fixes
...
Adds app name in HTML page titles on job web UI
2013-07-15 14:59:44 -07:00
Karen Feng
0d78b6d9cd
Links to job UI from standalone deploy cluster web UI: fixes SPARK-802
2013-07-15 13:47:38 -07:00
Karen Feng
b2aaa1199e
Adds app name in HTML page titles on job web UI: fixes SPARK-806
2013-07-15 11:44:42 -07:00
Matei Zaharia
4698a0d688
Shuffle ratings in a more efficient way at start of ALS
2013-07-15 02:54:11 +00:00
Matei Zaharia
d47c16f78d
Add an option to disable reference tracking in Kryo
2013-07-15 01:55:54 +00:00
Matei Zaharia
238d0e6893
Merge branch 'master' of github.com:mesos/spark
2013-07-15 00:30:32 +00:00
Matei Zaharia
ed7fd501cf
Make number of blocks in ALS configurable and lower the default
2013-07-15 00:30:10 +00:00
Matei Zaharia
c7877d5e16
Merge pull request #689 from BlackNiuza/application_status
...
Bug fix: SPARK-796
2013-07-14 12:58:13 -07:00
Matei Zaharia
10c05937bd
Merge pull request #699 from pwendell/ui-env
...
Add `Environment` tab to SparkUI.
2013-07-14 11:45:18 -07:00
Patrick Wendell
4883586838
Responding to Matei's review
2013-07-14 10:37:26 -07:00
BlackNiuza
00556a94c9
add spaces before curly braces and after for if conditions
2013-07-14 17:04:53 +08:00
Matei Zaharia
931e4c96ef
Fix a comment
2013-07-14 08:03:13 +00:00
Matei Zaharia
c5c38d1987
Some optimizations to loading phase of ALS
2013-07-14 07:59:50 +00:00
Matei Zaharia
b91a218cea
Cosmetic fixes to web UI
2013-07-14 07:31:33 +00:00
Matei Zaharia
a44a7b1238
Determine Spark core classes better in getCallSite
2013-07-14 07:23:09 +00:00
root
e271fde10b
Fixed a delay scheduling bug in the YARN branch, found by Patrick
2013-07-14 06:24:29 +00:00
Matei Zaharia
668b0dc6a7
Merge branch 'master' of github.com:mesos/spark
2013-07-13 19:10:46 -07:00
Matei Zaharia
cd28d9c147
Merge remote-tracking branch 'origin/pr/662'
...
Conflicts:
bin/compute-classpath.sh
2013-07-13 19:10:00 -07:00
Patrick Wendell
ddb97f0fdf
Add Environment
tab to SparkUI.
...
This adds a tab which displays system property and classpath information. This
can be useful in debugging various types of issues such as:
1. Extra/incorrect Hadoop jars being included in the classpath
2. Spark launching with a different JRE version than intended
3. Spark system properties not being set to intended values
4. User added jars that conflict with Spark jars
2013-07-13 16:14:40 -07:00
Matei Zaharia
89e8549f53
Merge pull request #698 from Reinvigorate/sm-deps-change
...
changing com.google.code.findbugs maven coordinates
2013-07-13 16:11:08 -07:00
Matei Zaharia
1889f9f450
Increase PermGen size
2013-07-13 14:43:21 -07:00
seanm
c4d5b01e44
changing com.google.code.findbugs maven coordinates
2013-07-13 14:56:23 -06:00
Matei Zaharia
77c69ae5a0
Merge pull request #697 from pwendell/block-locations
...
Show block locations in Web UI.
2013-07-12 23:05:21 -07:00
Matei Zaharia
5a7835c152
Merge pull request #691 from karenfeng/logpaging
...
Create log pages
2013-07-12 20:28:21 -07:00
Matei Zaharia
71ccca0cc1
Merge pull request #696 from woggle/executor-env
...
Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath.sh
2013-07-12 20:25:06 -07:00
Matei Zaharia
90fc3f30cd
Merge pull request #692 from Reinvigorate/takeOrdered
...
adding takeOrdered() to RDD
2013-07-12 20:23:36 -07:00
Patrick Wendell
08150f19ab
Minor style fix
2013-07-12 19:32:35 -07:00
Patrick Wendell
6855338e14
Show block locations in Web UI.
...
This fixes SPARK-769. Support is added for enumerating the locations of blocks
in the UI. There is also some minor cleanup in StorageUtils.
2013-07-12 19:30:32 -07:00
Karen Feng
73984b96a8
Removed unit test of nonexistent function Utils.lastNBytes
2013-07-12 14:26:56 -07:00
Charles Reiss
531a7e5574
Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath.
2013-07-12 12:58:25 -07:00