ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3654 commits 6 branches 181 tags 397 MiB
Commit graph

3654 commits

This branch
This branch
All branches
Author SHA1 Message Date
Xinghao 071afe2a33 New files from merge with master 2013-07-26 18:21:20 -07:00
Xinghao 10fd3949e6 Making ClassificationModel serializable 2013-07-26 17:49:11 -07:00
Xinghao f0a1f95228 Rename LogisticRegression, SVM and Lasso to *_LocalRandomSGD 2013-07-26 17:36:14 -07:00
Xinghao f74a03c6d8 Multiple changes 
- Changed LogisticRegression regularization parameter to 0
- Removed println from SVM predict function
- Fixed "Lasso" -> "SVM" in SVMGenerator
- Added comment in Updater.scala to indicate L1 regularization leads to
soft thresholding proximal function
 2013-07-26 17:29:44 -07:00
Karen Feng bd4cc52e30 Made metrics Option instead of Some, fixed NullPointerException 2013-07-26 17:23:18 -07:00
Reynold Xin f3d72ff2fe Merge pull request #739 from markhamstra/toolsPom 
Missing tools/pom.xml scalatest dependency
 2013-07-26 17:19:27 -07:00
Reynold Xin cb366774c8 Merge pull request #738 from harsha2010/pruning 
Fix bug in Partition Pruning.
 2013-07-26 16:59:30 -07:00
Mark Hamstra 3fc6408903 Added missing scalatest dependency 2013-07-26 16:10:20 -07:00
harshars 392d7474fd Code review 2013-07-26 15:23:15 -07:00
harshars 72cf7ec0e5 Indentation 2013-07-26 15:16:41 -07:00
harshars 822aac8f5a Indentation 2013-07-26 15:10:32 -07:00
harshars 743fc4e7aa Fix Bug in Partition Pruning, index of Pruned Partitions should inherit from parent 2013-07-26 14:35:17 -07:00
Karen Feng 3fbe9eaac0 Displys shuffle read/write only if exists, wraps if statements, trims old vals, grabs current time once 2013-07-26 11:51:38 -07:00
Karen Feng 22faeab261 Split Shuffle Activity overview column for read/write 2013-07-25 17:14:18 -07:00
Karen Feng d4bbc8bd25 Shows totals for shuffle data and CPU time in Stage, homepage overviews including active time 2013-07-25 15:59:52 -07:00
Matei Zaharia f3cf09491a Merge pull request #734 from woggle/executor-env2 
Get more env vars from driver rather than worker
 2013-07-25 14:53:21 -07:00
Charles Reiss a6de90c927 For standalone mode, get JAVA_HOME, SPARK_JAVA_OPTS, SPARK_LIBRARY_PATH from application env, not worker env 2013-07-25 12:42:30 -07:00
Matei Zaharia 8eb8b52997 Fix Chill version in Maven 2013-07-25 08:58:02 -07:00
Matei Zaharia e2421c1311 Update Chill reference in pom.xml too 2013-07-25 00:05:43 -07:00
Matei Zaharia 51c2427618 Merge pull request #732 from ryanlecompte/master 
Refactor Kryo serializer support to use chill/chill-java
 2013-07-25 00:03:11 -07:00
ryanlecompte e56aa75de0 fix wrapping 2013-07-24 22:08:09 -07:00
ryanlecompte 30a369a808 update pom.xml 2013-07-24 20:55:48 -07:00
ryanlecompte fc4b025314 add test 2013-07-24 20:53:15 -07:00
ryanlecompte a1c515fb02 add copyright back in 2013-07-24 20:50:32 -07:00
ryanlecompte 8e0939f5a9 refactor Kryo serializer support to use chill/chill-java 2013-07-24 20:43:57 -07:00
Matei Zaharia c258718606 Fix Maven build errors after previous commits 2013-07-24 16:12:32 -07:00
Xinghao eef678703e Adding SVM and Lasso, moving LogisticRegression to classification from regression 
Also, add regularization parameter to SGD
 2013-07-24 15:32:50 -07:00
Matei Zaharia 52723b99e4 Merge pull request #728 from jey/examples-jar-env 
Fix setting of SPARK_EXAMPLES_JAR
 2013-07-24 14:33:02 -07:00
Matei Zaharia 20338c2983 Merge pull request #729 from karenfeng/ui-811 
Stage Page updates
 2013-07-24 14:32:24 -07:00
Jey Kottalam 1d10192806 Fix setting of SPARK_EXAMPLES_JAR 2013-07-24 14:04:17 -07:00
Karen Feng 57009eef90 Fixed consistency of "success" status string 2013-07-24 13:43:09 -07:00
Karen Feng 4280e1768d Removed finished status for task info, changed name of success case 2013-07-24 12:48:48 -07:00
Matei Zaharia 5584ebcbd3 Merge pull request #675 from c0s/assembly 
Building spark assembly for further consumption of the Spark project with a deployed cluster
 2013-07-24 11:46:46 -07:00
Karen Feng bd3931c874 Changed ifs with returns to if/else 2013-07-24 11:27:17 -07:00
Karen Feng 93c6015f82 Shows task status and running tasks on Stage Page: fixes SPARK-804 and 811 2013-07-24 10:53:02 -07:00
Matei Zaharia a73f3ee536 Merge pull request #671 from jerryshao/master 
Add metrics system for Spark
 2013-07-24 08:59:14 -07:00
jerryshao 31ec72b243 Code refactor according to comments 2013-07-24 14:57:47 +08:00
jerryshao 8d1ef7f2df Code style changes 2013-07-24 14:57:47 +08:00
Andrew xia 05637de842 Change class xxxInstrumentation to class xxxSource 2013-07-24 14:57:47 +08:00
Andrew xia 5b4a2f2094 Add metrics config template file 2013-07-24 14:57:47 +08:00
Andrew xia ed1a3bc206 continue to refactor code style and functions 2013-07-24 14:57:47 +08:00
jerryshao 5730193e0c Fix some typos 2013-07-24 14:57:47 +08:00
jerryshao a79f6077f0 Add Maven metrics library dependency and code changes 2013-07-24 14:57:47 +08:00
jerryshao 1daff54b2e Change Executor MetricsSystem initialize code to SparkEnv 2013-07-24 14:57:47 +08:00
Andrew xia 5f8802c1fb Register and init metricsSystem in SparkContext 
Conflicts:

	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/SparkEnv.scala
 2013-07-24 14:57:47 +08:00
Andrew xia 9cea0c2818 Refactor metricsSystem unit test, add resource files. 2013-07-24 14:57:47 +08:00
Andrew xia 7d2eada451 Add metrics source of DAGScheduler and blockManager 
Conflicts:

	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/SparkEnv.scala
 2013-07-24 14:57:47 +08:00
jerryshao e9ac88754d Remove twice add Source bug and code clean 2013-07-24 14:57:47 +08:00
jerryshao e080588f73 Add metrics system unit test 2013-07-24 14:57:47 +08:00
jerryshao 5ce5dc9fcd Add default properties to deal with no configure file situation 2013-07-24 14:57:47 +08:00