Liang-Chi Hsieh
4530e8a9bf
fix typo.
2013-07-20 00:04:25 +08:00
Liang-Chi Hsieh
aa6f83289b
A better fix for giving local jars unde Yarn mode.
2013-07-19 22:25:28 +08:00
Liang-Chi Hsieh
a613628c50
Do not copy local jars given to SparkContext in yarn mode since the Context is not running on local. This bug causes failure when jars can not be found. Example codes (such as spark.examples.SparkPi) can not work without this fix under yarn mode.
2013-07-19 16:59:12 +08:00
Matei Zaharia
0d0a47cb4f
Merge pull request #710 from shivaram/ml-updates
...
Updates to LogisticRegression
2013-07-18 12:06:37 -07:00
Shivaram Venkataraman
2c9ea56db4
Rename classes to be called DataGenerator
2013-07-18 11:57:14 -07:00
Shivaram Venkataraman
7ab1170503
Refactor data generators to have a function that can be used in tests.
2013-07-18 11:55:19 -07:00
Matei Zaharia
c6235b51ba
Merge pull request #714 from adatao/master
...
[BUGFIX] Fix for sbt/sbt script SPARK_HOME setting
2013-07-18 11:43:48 -07:00
Matei Zaharia
009c79e5a5
Merge pull request #715 from viirya/dev1
...
fix a bug in build process that pulls in two versions of ASM.
2013-07-18 11:41:52 -07:00
Liang-Chi Hsieh
3aad452653
fix a bug in build process that pulls in two versionf of ASM.
2013-07-19 02:29:46 +08:00
Matei Zaharia
985a9e324a
Merge pull request #712 from stayhf/SPARK-817
...
Consistently invoke bash with /usr/bin/env bash in scripts to make code ...
2013-07-17 22:27:19 -07:00
Ubuntu
88a0823c58
Consistently invoke bash with /usr/bin/env bash in scripts to make code more portable (JIRA Ticket SPARK-817)
2013-07-18 00:51:18 +00:00
Shivaram Venkataraman
217667174e
Return Array[Double] from SGD instead of DoubleMatrix
2013-07-17 16:08:34 -07:00
Shivaram Venkataraman
45f3c85518
Change weights to be Array[Double] in LR model.
...
Also ensure weights are initialized to a column vector.
2013-07-17 16:03:29 -07:00
Shivaram Venkataraman
3bf9897136
Rename loss -> stochasticLoss and add a note to explain why we have
...
multiple train methods.
2013-07-17 14:20:24 -07:00
Shivaram Venkataraman
64b88e039a
Move ML lib data generator files to util/
2013-07-17 14:11:44 -07:00
Shivaram Venkataraman
84fa20c2a1
Allow initial weight vectors in LogisticRegression.
...
Also move LogisticGradient to the LogisticRegression file and fix the
unit tests log path.
2013-07-17 14:04:05 -07:00
ctn
a1d2c34346
[BUGFIX] Fix for sbt/sbt script SPARK_HOME setting
...
In some environments, this command
export SPARK_HOME=$(cd "$(dirname $0)/.."; pwd)
echoes two paths, one by the "cd ..", and one by the "pwd". Note the resulting
erroneous -jar paths below:
ctn@ubuntu:~/src/spark$ sbt/sbt
+ EXTRA_ARGS=
+ '[' '' '!=' '' ']'
+++ dirname sbt/sbt
++ cd sbt/..
++ pwd
+ export 'SPARK_HOME=/home/ctn/src/spark
/home/ctn/src/spark'
+ SPARK_HOME='/home/ctn/src/spark
/home/ctn/src/spark'
+ export SPARK_TESTING=1
+ SPARK_TESTING=1
+ java -Xmx1200m -XX:MaxPermSize=350m -XX:ReservedCodeCacheSize=128m -jar /home/ctn/src/spark /home/ctn/src/spark/sbt/sbt-launch-0.11.3-2.jar
Error: Invalid or corrupt jarfile /home/ctn/src/spark
Committer: ctn <ctn@adatao.com>
On branch master
Changes to be committed:
- Send output of the "cd .." part to /dev/null
modified: sbt/sbt
2013-07-17 11:27:11 -07:00
ctn
b29c53e6aa
Merge remote-tracking branch 'upstream/master'
2013-07-17 11:24:03 -07:00
Matei Zaharia
cad48edb70
Merge pull request #708 from ScrapCodes/dependencies-upgrade
...
Dependency upgrade Akka 2.0.3 -> 2.0.5
2013-07-16 21:41:28 -07:00
Matei Zaharia
7e74ab5b0f
Merge remote-tracking branch 'origin/pr/704'
...
Conflicts:
make-distribution.sh
2013-07-16 19:13:07 -07:00
Matei Zaharia
4ff494de20
Some missing license headers
2013-07-16 17:26:48 -07:00
Matei Zaharia
af3c9d5042
Add Apache license headers and LICENSE and NOTICE files
2013-07-16 17:21:33 -07:00
Matei Zaharia
00a14deb6d
Update to latest Scala Maven plugin and allow Zinc external compiler
2013-07-16 11:52:20 -07:00
Matei Zaharia
b1f9f64743
Merge branch 'master' of github.com:mesos/spark
2013-07-16 11:01:53 -07:00
Matei Zaharia
5c388808a8
SPARK-814: Result stages should be named after action
2013-07-16 11:01:14 -07:00
Matei Zaharia
f347cc3f65
Fix deprecation warning and style issues
2013-07-16 10:53:30 -07:00
Prashant Sharma
2748e73eb9
Dependency upgrade Akka 2.0.3 -> 2.0.5
2013-07-16 16:08:46 +05:30
Matei Zaharia
8a8a8f2de2
Merge pull request #705 from rxin/errormessages
...
Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions.
2013-07-15 23:09:21 -07:00
Reynold Xin
69316603d6
Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions.
2013-07-15 22:50:11 -07:00
seanm
90b0142985
adding files generated from make-distribution.sh to .gitignore
2013-07-15 19:13:39 -06:00
seanm
a96b4ef761
dding tgz option to make-distribution.sh
2013-07-15 19:13:17 -06:00
Matei Zaharia
ed8415b2fa
Merge pull request #703 from karenfeng/ui-802
...
Link to job UI from standalone deploy cluster web UI
2013-07-15 16:41:04 -07:00
Karen Feng
6dc7c9bfb1
Removed job UI column, linked description to job UI
2013-07-15 16:33:50 -07:00
Karen Feng
fbf5aa761e
Removed log message, added field in master UI to link to log UI
2013-07-15 15:50:03 -07:00
Karen Feng
eac381a957
Merge branch 'ui-802' of https://github.com/karenfeng/spark into ui-802
2013-07-15 15:48:44 -07:00
Karen Feng
3955711250
Added field to master UI with link to job UI
2013-07-15 15:47:21 -07:00
Matei Zaharia
e3d3e6f0ab
Merge pull request #702 from karenfeng/ui-fixes
...
Adds app name in HTML page titles on job web UI
2013-07-15 14:59:44 -07:00
Karen Feng
0d78b6d9cd
Links to job UI from standalone deploy cluster web UI: fixes SPARK-802
2013-07-15 13:47:38 -07:00
Karen Feng
b2aaa1199e
Adds app name in HTML page titles on job web UI: fixes SPARK-806
2013-07-15 11:44:42 -07:00
Matei Zaharia
4698a0d688
Shuffle ratings in a more efficient way at start of ALS
2013-07-15 02:54:11 +00:00
Matei Zaharia
d47c16f78d
Add an option to disable reference tracking in Kryo
2013-07-15 01:55:54 +00:00
Matei Zaharia
238d0e6893
Merge branch 'master' of github.com:mesos/spark
2013-07-15 00:30:32 +00:00
Matei Zaharia
ed7fd501cf
Make number of blocks in ALS configurable and lower the default
2013-07-15 00:30:10 +00:00
Matei Zaharia
c7877d5e16
Merge pull request #689 from BlackNiuza/application_status
...
Bug fix: SPARK-796
2013-07-14 12:58:13 -07:00
Matei Zaharia
10c05937bd
Merge pull request #699 from pwendell/ui-env
...
Add `Environment` tab to SparkUI.
2013-07-14 11:45:18 -07:00
Patrick Wendell
4883586838
Responding to Matei's review
2013-07-14 10:37:26 -07:00
BlackNiuza
00556a94c9
add spaces before curly braces and after for if conditions
2013-07-14 17:04:53 +08:00
Matei Zaharia
931e4c96ef
Fix a comment
2013-07-14 08:03:13 +00:00
Matei Zaharia
c5c38d1987
Some optimizations to loading phase of ALS
2013-07-14 07:59:50 +00:00
Matei Zaharia
b91a218cea
Cosmetic fixes to web UI
2013-07-14 07:31:33 +00:00