Karen Feng
636b19f833
Merge branch 'master' of https://github.com/mesos/spark into ui-808
2013-07-22 09:53:26 -07:00
Evan Chan
0337d88321
Add a public method getCachedRdds to SparkContext
2013-07-21 18:26:14 -07:00
Konstantin Boudnik
f4d514810e
Building spark assembly for further consumption of the Spark project with a deployed cluster
2013-07-21 11:47:29 -07:00
Matei Zaharia
15fb394833
Merge pull request #716 from c0s/webui-port
...
Regression: default webui-port can't be set via command line "--webui-port" anymore
2013-07-21 10:33:38 -07:00
Karen Feng
865dc63bac
Changed table format for executors
2013-07-19 15:57:01 -07:00
Karen Feng
81bb5dc640
Creates Executors tab for application with RDD block and memory/disk used, solves SPARK-808
2013-07-19 14:08:30 -07:00
Konstantin Boudnik
cfce9a6a36
Regression: default webui-port can't be set via command line "--webui-port" anymore
2013-07-19 14:00:58 -07:00
Matei Zaharia
c40f0f21f1
Merge pull request #711 from shivaram/ml-generators
...
Move ML lib data generator files to util/
2013-07-19 13:33:04 -07:00
Matei Zaharia
413b84172e
Merge pull request #717 from viirya/dev1
...
Do not copy local jars given to SparkContext in yarn mode
2013-07-19 13:31:38 -07:00
Liang-Chi Hsieh
d1738d72ba
also exclude asm for hadoop2. hadoop1 looks like no need to do that too.
2013-07-20 00:37:24 +08:00
Liang-Chi Hsieh
4530e8a9bf
fix typo.
2013-07-20 00:04:25 +08:00
Liang-Chi Hsieh
aa6f83289b
A better fix for giving local jars unde Yarn mode.
2013-07-19 22:25:28 +08:00
Liang-Chi Hsieh
a613628c50
Do not copy local jars given to SparkContext in yarn mode since the Context is not running on local. This bug causes failure when jars can not be found. Example codes (such as spark.examples.SparkPi) can not work without this fix under yarn mode.
2013-07-19 16:59:12 +08:00
Matei Zaharia
0d0a47cb4f
Merge pull request #710 from shivaram/ml-updates
...
Updates to LogisticRegression
2013-07-18 12:06:37 -07:00
Shivaram Venkataraman
2c9ea56db4
Rename classes to be called DataGenerator
2013-07-18 11:57:14 -07:00
Shivaram Venkataraman
7ab1170503
Refactor data generators to have a function that can be used in tests.
2013-07-18 11:55:19 -07:00
Matei Zaharia
c6235b51ba
Merge pull request #714 from adatao/master
...
[BUGFIX] Fix for sbt/sbt script SPARK_HOME setting
2013-07-18 11:43:48 -07:00
Matei Zaharia
009c79e5a5
Merge pull request #715 from viirya/dev1
...
fix a bug in build process that pulls in two versions of ASM.
2013-07-18 11:41:52 -07:00
Liang-Chi Hsieh
3aad452653
fix a bug in build process that pulls in two versionf of ASM.
2013-07-19 02:29:46 +08:00
Matei Zaharia
985a9e324a
Merge pull request #712 from stayhf/SPARK-817
...
Consistently invoke bash with /usr/bin/env bash in scripts to make code ...
2013-07-17 22:27:19 -07:00
Ubuntu
88a0823c58
Consistently invoke bash with /usr/bin/env bash in scripts to make code more portable (JIRA Ticket SPARK-817)
2013-07-18 00:51:18 +00:00
Patrick Wendell
31b3c1df54
Small improvement in python script
2013-07-17 17:43:15 -07:00
Shivaram Venkataraman
217667174e
Return Array[Double] from SGD instead of DoubleMatrix
2013-07-17 16:08:34 -07:00
Shivaram Venkataraman
45f3c85518
Change weights to be Array[Double] in LR model.
...
Also ensure weights are initialized to a column vector.
2013-07-17 16:03:29 -07:00
Shivaram Venkataraman
3bf9897136
Rename loss -> stochasticLoss and add a note to explain why we have
...
multiple train methods.
2013-07-17 14:20:24 -07:00
Shivaram Venkataraman
64b88e039a
Move ML lib data generator files to util/
2013-07-17 14:11:44 -07:00
Shivaram Venkataraman
84fa20c2a1
Allow initial weight vectors in LogisticRegression.
...
Also move LogisticGradient to the LogisticRegression file and fix the
unit tests log path.
2013-07-17 14:04:05 -07:00
ctn
a1d2c34346
[BUGFIX] Fix for sbt/sbt script SPARK_HOME setting
...
In some environments, this command
export SPARK_HOME=$(cd "$(dirname $0)/.."; pwd)
echoes two paths, one by the "cd ..", and one by the "pwd". Note the resulting
erroneous -jar paths below:
ctn@ubuntu:~/src/spark$ sbt/sbt
+ EXTRA_ARGS=
+ '[' '' '!=' '' ']'
+++ dirname sbt/sbt
++ cd sbt/..
++ pwd
+ export 'SPARK_HOME=/home/ctn/src/spark
/home/ctn/src/spark'
+ SPARK_HOME='/home/ctn/src/spark
/home/ctn/src/spark'
+ export SPARK_TESTING=1
+ SPARK_TESTING=1
+ java -Xmx1200m -XX:MaxPermSize=350m -XX:ReservedCodeCacheSize=128m -jar /home/ctn/src/spark /home/ctn/src/spark/sbt/sbt-launch-0.11.3-2.jar
Error: Invalid or corrupt jarfile /home/ctn/src/spark
Committer: ctn <ctn@adatao.com>
On branch master
Changes to be committed:
- Send output of the "cd .." part to /dev/null
modified: sbt/sbt
2013-07-17 11:27:11 -07:00
ctn
b29c53e6aa
Merge remote-tracking branch 'upstream/master'
2013-07-17 11:24:03 -07:00
Matei Zaharia
cad48edb70
Merge pull request #708 from ScrapCodes/dependencies-upgrade
...
Dependency upgrade Akka 2.0.3 -> 2.0.5
2013-07-16 21:41:28 -07:00
Matei Zaharia
7e74ab5b0f
Merge remote-tracking branch 'origin/pr/704'
...
Conflicts:
make-distribution.sh
2013-07-16 19:13:07 -07:00
Matei Zaharia
4ff494de20
Some missing license headers
2013-07-16 17:26:48 -07:00
Matei Zaharia
af3c9d5042
Add Apache license headers and LICENSE and NOTICE files
2013-07-16 17:21:33 -07:00
Matei Zaharia
00a14deb6d
Update to latest Scala Maven plugin and allow Zinc external compiler
2013-07-16 11:52:20 -07:00
Matei Zaharia
b1f9f64743
Merge branch 'master' of github.com:mesos/spark
2013-07-16 11:01:53 -07:00
Matei Zaharia
5c388808a8
SPARK-814: Result stages should be named after action
2013-07-16 11:01:14 -07:00
Matei Zaharia
f347cc3f65
Fix deprecation warning and style issues
2013-07-16 10:53:30 -07:00
Prashant Sharma
2748e73eb9
Dependency upgrade Akka 2.0.3 -> 2.0.5
2013-07-16 16:08:46 +05:30
Matei Zaharia
8a8a8f2de2
Merge pull request #705 from rxin/errormessages
...
Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions.
2013-07-15 23:09:21 -07:00
Reynold Xin
69316603d6
Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions.
2013-07-15 22:50:11 -07:00
seanm
90b0142985
adding files generated from make-distribution.sh to .gitignore
2013-07-15 19:13:39 -06:00
seanm
a96b4ef761
dding tgz option to make-distribution.sh
2013-07-15 19:13:17 -06:00
Matei Zaharia
ed8415b2fa
Merge pull request #703 from karenfeng/ui-802
...
Link to job UI from standalone deploy cluster web UI
2013-07-15 16:41:04 -07:00
Karen Feng
6dc7c9bfb1
Removed job UI column, linked description to job UI
2013-07-15 16:33:50 -07:00
Karen Feng
fbf5aa761e
Removed log message, added field in master UI to link to log UI
2013-07-15 15:50:03 -07:00
Karen Feng
eac381a957
Merge branch 'ui-802' of https://github.com/karenfeng/spark into ui-802
2013-07-15 15:48:44 -07:00
Karen Feng
3955711250
Added field to master UI with link to job UI
2013-07-15 15:47:21 -07:00
Matei Zaharia
e3d3e6f0ab
Merge pull request #702 from karenfeng/ui-fixes
...
Adds app name in HTML page titles on job web UI
2013-07-15 14:59:44 -07:00
Karen Feng
0d78b6d9cd
Links to job UI from standalone deploy cluster web UI: fixes SPARK-802
2013-07-15 13:47:38 -07:00
Karen Feng
b2aaa1199e
Adds app name in HTML page titles on job web UI: fixes SPARK-806
2013-07-15 11:44:42 -07:00