ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3337 commits 6 branches 181 tags 397 MiB
Commit graph

3337 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia f11ad72d4e Some fixes to Python examples (style and package name for LR) 2013-07-27 21:12:22 -04:00
Reynold Xin f3d72ff2fe Merge pull request #739 from markhamstra/toolsPom 
Missing tools/pom.xml scalatest dependency
 2013-07-26 17:19:27 -07:00
Reynold Xin cb366774c8 Merge pull request #738 from harsha2010/pruning 
Fix bug in Partition Pruning.
 2013-07-26 16:59:30 -07:00
Mark Hamstra 3fc6408903 Added missing scalatest dependency 2013-07-26 16:10:20 -07:00
harshars 392d7474fd Code review 2013-07-26 15:23:15 -07:00
harshars 72cf7ec0e5 Indentation 2013-07-26 15:16:41 -07:00
harshars 822aac8f5a Indentation 2013-07-26 15:10:32 -07:00
harshars 743fc4e7aa Fix Bug in Partition Pruning, index of Pruned Partitions should inherit from parent 2013-07-26 14:35:17 -07:00
Matei Zaharia f3cf09491a Merge pull request #734 from woggle/executor-env2 
Get more env vars from driver rather than worker
 2013-07-25 14:53:21 -07:00
Charles Reiss a6de90c927 For standalone mode, get JAVA_HOME, SPARK_JAVA_OPTS, SPARK_LIBRARY_PATH from application env, not worker env 2013-07-25 12:42:30 -07:00
Matei Zaharia 8eb8b52997 Fix Chill version in Maven 2013-07-25 08:58:02 -07:00
Matei Zaharia e2421c1311 Update Chill reference in pom.xml too 2013-07-25 00:05:43 -07:00
Matei Zaharia 51c2427618 Merge pull request #732 from ryanlecompte/master 
Refactor Kryo serializer support to use chill/chill-java
 2013-07-25 00:03:11 -07:00
ryanlecompte e56aa75de0 fix wrapping 2013-07-24 22:08:09 -07:00
ryanlecompte 30a369a808 update pom.xml 2013-07-24 20:55:48 -07:00
ryanlecompte fc4b025314 add test 2013-07-24 20:53:15 -07:00
ryanlecompte a1c515fb02 add copyright back in 2013-07-24 20:50:32 -07:00
ryanlecompte 8e0939f5a9 refactor Kryo serializer support to use chill/chill-java 2013-07-24 20:43:57 -07:00
Matei Zaharia c258718606 Fix Maven build errors after previous commits 2013-07-24 16:12:32 -07:00
Matei Zaharia 52723b99e4 Merge pull request #728 from jey/examples-jar-env 
Fix setting of SPARK_EXAMPLES_JAR
 2013-07-24 14:33:02 -07:00
Matei Zaharia 20338c2983 Merge pull request #729 from karenfeng/ui-811 
Stage Page updates
 2013-07-24 14:32:24 -07:00
Jey Kottalam 1d10192806 Fix setting of SPARK_EXAMPLES_JAR 2013-07-24 14:04:17 -07:00
Karen Feng 57009eef90 Fixed consistency of "success" status string 2013-07-24 13:43:09 -07:00
Karen Feng 4280e1768d Removed finished status for task info, changed name of success case 2013-07-24 12:48:48 -07:00
Matei Zaharia 5584ebcbd3 Merge pull request #675 from c0s/assembly 
Building spark assembly for further consumption of the Spark project with a deployed cluster
 2013-07-24 11:46:46 -07:00
Karen Feng bd3931c874 Changed ifs with returns to if/else 2013-07-24 11:27:17 -07:00
Karen Feng 93c6015f82 Shows task status and running tasks on Stage Page: fixes SPARK-804 and 811 2013-07-24 10:53:02 -07:00
Matei Zaharia a73f3ee536 Merge pull request #671 from jerryshao/master 
Add metrics system for Spark
 2013-07-24 08:59:14 -07:00
jerryshao 31ec72b243 Code refactor according to comments 2013-07-24 14:57:47 +08:00
jerryshao 8d1ef7f2df Code style changes 2013-07-24 14:57:47 +08:00
Andrew xia 05637de842 Change class xxxInstrumentation to class xxxSource 2013-07-24 14:57:47 +08:00
Andrew xia 5b4a2f2094 Add metrics config template file 2013-07-24 14:57:47 +08:00
Andrew xia ed1a3bc206 continue to refactor code style and functions 2013-07-24 14:57:47 +08:00
jerryshao 5730193e0c Fix some typos 2013-07-24 14:57:47 +08:00
jerryshao a79f6077f0 Add Maven metrics library dependency and code changes 2013-07-24 14:57:47 +08:00
jerryshao 1daff54b2e Change Executor MetricsSystem initialize code to SparkEnv 2013-07-24 14:57:47 +08:00
Andrew xia 5f8802c1fb Register and init metricsSystem in SparkContext 
Conflicts:

	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/SparkEnv.scala
 2013-07-24 14:57:47 +08:00
Andrew xia 9cea0c2818 Refactor metricsSystem unit test, add resource files. 2013-07-24 14:57:47 +08:00
Andrew xia 7d2eada451 Add metrics source of DAGScheduler and blockManager 
Conflicts:

	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/SparkEnv.scala
 2013-07-24 14:57:47 +08:00
jerryshao e9ac88754d Remove twice add Source bug and code clean 2013-07-24 14:57:47 +08:00
jerryshao e080588f73 Add metrics system unit test 2013-07-24 14:57:47 +08:00
jerryshao 5ce5dc9fcd Add default properties to deal with no configure file situation 2013-07-24 14:57:47 +08:00
jerryshao 871bc1687e Add Executor instrumentation 2013-07-24 14:57:46 +08:00
jerryshao 576528f0f9 Add dependency of Codahale's metrics library 2013-07-24 14:57:46 +08:00
jerryshao 7fb574bf66 Code clean and remarshal 2013-07-24 14:57:46 +08:00
Andrew xia 4d6dd67fa1 refactor metrics system 
1.change source abstract class to support MetricRegistry
2.change master/work/jvm source class
 2013-07-24 14:57:46 +08:00
jerryshao 03f9871116 MetricsSystem refactor 2013-07-24 14:57:46 +08:00
jerryshao c3daad3f65 Update metric source support for instrumentation 2013-07-24 14:57:46 +08:00
jerryshao 9dec8c73e6 Add Master and Worker instrumentation support 2013-07-24 14:57:46 +08:00
jerryshao 503acd3a37 Build metrics system framwork 2013-07-24 14:57:46 +08:00