Matei Zaharia
d75c308695
Use None instead of empty string as it's slightly smaller/faster
2013-07-29 02:51:43 -04:00
Matei Zaharia
96b50e82dc
Allow python/run-tests to run from any directory
2013-07-29 02:51:43 -04:00
Matei Zaharia
b5ec355622
Optimize Python foreach() to not return as many objects
2013-07-29 02:51:43 -04:00
Matei Zaharia
b9d6783f36
Optimize Python take() to not compute entire first partition
2013-07-29 02:51:43 -04:00
Xinghao
c823ee1e2b
Replace map-reduce with dot operator using DoubleMatrix
2013-07-28 22:17:53 -07:00
Xinghao
96e04f4cb7
Fixed SVM and LR train functions to take Int instead of Double for Classification
2013-07-28 22:12:39 -07:00
Xinghao
9398dced03
Changed Classification to return Int instead of Double
...
Also minor changes to formatting and comments
2013-07-28 21:39:19 -07:00
Xinghao
67de051bbb
SVMSuite and LassoSuite rewritten to follow closely with LogisticRegressionSuite
2013-07-28 21:09:56 -07:00
Xinghao
29e042940a
Move data generators to util
2013-07-28 20:39:52 -07:00
Matei Zaharia
72ff62a37c
Two fixes to IPython support:
...
- Don't attempt to run worker processes with ipython (that can cause
some crashes as ipython prints things to standard out)
- Allow passing some IPYTHON_OPTS to launch things like the notebook
2013-07-28 22:23:13 -04:00
Xinghao
ccfa362dde
Change *_LocalRandomSGD to *LocalRandomSGD
2013-07-28 10:33:57 -07:00
Dmitriy Lyubimov
0862494d44
typo
2013-07-27 23:16:20 -07:00
Dmitriy Lyubimov
f5067abe85
changes per comments.
2013-07-27 23:08:00 -07:00
Dmitriy Lyubimov
b241fcfb35
reverting changes to SparkBuild.scala
2013-07-27 22:53:01 -07:00
Matei Zaharia
f11ad72d4e
Some fixes to Python examples (style and package name for LR)
2013-07-27 21:12:22 -04:00
Karen Feng
077f2dad22
Fixed outdated bugs
2013-07-27 16:39:36 -07:00
Patrick Wendell
bcafb36c1e
Slight wording change
2013-07-27 16:03:50 -07:00
Patrick Wendell
8177165ac4
Log executor on finish
2013-07-27 16:02:06 -07:00
Patrick Wendell
c2223e6801
Improve catch scope and logging for client stop()
...
This does two things:
1. Catches the more general `TimeoutException`, since those can be thrown.
2. Logs at info level when a timeout is detected.
2013-07-27 16:02:06 -07:00
Karen Feng
5a93e3c58c
Cleaned up code based on pwendell's suggestions
2013-07-27 15:55:26 -07:00
Karen Feng
dcc4743a95
Moved val now to render
2013-07-27 12:52:53 -07:00
Karen Feng
1714693324
Current time called once with value now
2013-07-27 12:24:41 -07:00
Dmitriy Lyubimov
6a47cee721
style
2013-07-26 22:35:13 -07:00
Dmitriy Lyubimov
0c391feb73
Maximum task failures configurable
2013-07-26 22:34:43 -07:00
Dmitriy Lyubimov
23f3e0f117
mixing in SharedSparkContext for the kryo-collect test
2013-07-26 19:15:11 -07:00
Xinghao
b0bbc7f6a8
Resolve conflicts with master, removed regParam for LogisticRegression
2013-07-26 18:57:39 -07:00
Xinghao
071afe2a33
New files from merge with master
2013-07-26 18:21:20 -07:00
Xinghao
10fd3949e6
Making ClassificationModel serializable
2013-07-26 17:49:11 -07:00
Xinghao
f0a1f95228
Rename LogisticRegression, SVM and Lasso to *_LocalRandomSGD
2013-07-26 17:36:14 -07:00
Xinghao
f74a03c6d8
Multiple changes
...
- Changed LogisticRegression regularization parameter to 0
- Removed println from SVM predict function
- Fixed "Lasso" -> "SVM" in SVMGenerator
- Added comment in Updater.scala to indicate L1 regularization leads to
soft thresholding proximal function
2013-07-26 17:29:44 -07:00
Karen Feng
bd4cc52e30
Made metrics Option instead of Some, fixed NullPointerException
2013-07-26 17:23:18 -07:00
Reynold Xin
f3d72ff2fe
Merge pull request #739 from markhamstra/toolsPom
...
Missing tools/pom.xml scalatest dependency
2013-07-26 17:19:27 -07:00
Reynold Xin
cb366774c8
Merge pull request #738 from harsha2010/pruning
...
Fix bug in Partition Pruning.
2013-07-26 16:59:30 -07:00
Mark Hamstra
3fc6408903
Added missing scalatest dependency
2013-07-26 16:10:20 -07:00
harshars
392d7474fd
Code review
2013-07-26 15:23:15 -07:00
harshars
72cf7ec0e5
Indentation
2013-07-26 15:16:41 -07:00
harshars
822aac8f5a
Indentation
2013-07-26 15:10:32 -07:00
harshars
743fc4e7aa
Fix Bug in Partition Pruning, index of Pruned Partitions should inherit from parent
2013-07-26 14:35:17 -07:00
Karen Feng
3fbe9eaac0
Displys shuffle read/write only if exists, wraps if statements, trims old vals, grabs current time once
2013-07-26 11:51:38 -07:00
Karen Feng
22faeab261
Split Shuffle Activity overview column for read/write
2013-07-25 17:14:18 -07:00
Karen Feng
d4bbc8bd25
Shows totals for shuffle data and CPU time in Stage, homepage overviews including active time
2013-07-25 15:59:52 -07:00
Matei Zaharia
f3cf09491a
Merge pull request #734 from woggle/executor-env2
...
Get more env vars from driver rather than worker
2013-07-25 14:53:21 -07:00
Charles Reiss
a6de90c927
For standalone mode, get JAVA_HOME, SPARK_JAVA_OPTS, SPARK_LIBRARY_PATH from application env, not worker env
2013-07-25 12:42:30 -07:00
Matei Zaharia
8eb8b52997
Fix Chill version in Maven
2013-07-25 08:58:02 -07:00
Matei Zaharia
e2421c1311
Update Chill reference in pom.xml too
2013-07-25 00:05:43 -07:00
Matei Zaharia
51c2427618
Merge pull request #732 from ryanlecompte/master
...
Refactor Kryo serializer support to use chill/chill-java
2013-07-25 00:03:11 -07:00
ryanlecompte
e56aa75de0
fix wrapping
2013-07-24 22:08:09 -07:00
ryanlecompte
30a369a808
update pom.xml
2013-07-24 20:55:48 -07:00
ryanlecompte
fc4b025314
add test
2013-07-24 20:53:15 -07:00
ryanlecompte
a1c515fb02
add copyright back in
2013-07-24 20:50:32 -07:00