ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3266 commits 6 branches 181 tags 397 MiB
Commit graph

1510 commits

Author SHA1 Message Date
Karen Feng 9f2dbb2a7c Adds/removes active tasks only once 2013-07-23 15:10:09 -07:00
Karen Feng 0200801a55 Tracks task start events and shows number of active tasks on Executor UI 2013-07-23 13:35:43 -07:00
Matei Zaharia 401aac8b18 Merge pull request #719 from karenfeng/ui-808 
Creates Executors tab for Jobs UI
 2013-07-22 16:57:16 -07:00
Karen Feng 872c97ad82 Split task columns, memory columns sort by numeric value 2013-07-22 16:54:37 -07:00
Karen Feng 2eea974795 Executors UI now calls executor ID from TaskInfo instead of TaskMetrics 2013-07-22 15:15:54 -07:00
Karen Feng 85c4d7bf3b Shows number of complete/total/failed tasks (bug: failed tasks assigned to null executor) 2013-07-22 14:35:47 -07:00
Josh Rosen f649dabb4a Fix bug: DoubleRDDFunctions.sampleStdev() computed non-sample stdev(). 
Update JavaDoubleRDD to add new methods and docs.

Fixes SPARK-825.
 2013-07-22 13:21:48 -07:00
Karen Feng 8901f379c9 Fixed memory used/remaining/total bug 2013-07-22 09:58:03 -07:00
Karen Feng 636b19f833 Merge branch 'master' of https://github.com/mesos/spark into ui-808 2013-07-22 09:53:26 -07:00
Karen Feng 865dc63bac Changed table format for executors 2013-07-19 15:57:01 -07:00
Karen Feng 81bb5dc640 Creates Executors tab for application with RDD block and memory/disk used, solves SPARK-808 2013-07-19 14:08:30 -07:00
Konstantin Boudnik cfce9a6a36 Regression: default webui-port can't be set via command line "--webui-port" anymore 2013-07-19 14:00:58 -07:00
Liang-Chi Hsieh aa6f83289b A better fix for giving local jars unde Yarn mode. 2013-07-19 22:25:28 +08:00
Liang-Chi Hsieh a613628c50 Do not copy local jars given to SparkContext in yarn mode since the Context is not running on local. This bug causes failure when jars can not be found. Example codes (such as spark.examples.SparkPi) can not work without this fix under yarn mode. 2013-07-19 16:59:12 +08:00
Matei Zaharia af3c9d5042 Add Apache license headers and LICENSE and NOTICE files 2013-07-16 17:21:33 -07:00
Matei Zaharia b1f9f64743 Merge branch 'master' of github.com:mesos/spark 2013-07-16 11:01:53 -07:00
Matei Zaharia 5c388808a8 SPARK-814: Result stages should be named after action 2013-07-16 11:01:14 -07:00
Reynold Xin 69316603d6 Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions. 2013-07-15 22:50:11 -07:00
Karen Feng 6dc7c9bfb1 Removed job UI column, linked description to job UI 2013-07-15 16:33:50 -07:00
Karen Feng fbf5aa761e Removed log message, added field in master UI to link to log UI 2013-07-15 15:50:03 -07:00
Karen Feng eac381a957 Merge branch 'ui-802' of https://github.com/karenfeng/spark into ui-802 2013-07-15 15:48:44 -07:00
Karen Feng 3955711250 Added field to master UI with link to job UI 2013-07-15 15:47:21 -07:00
Karen Feng 0d78b6d9cd Links to job UI from standalone deploy cluster web UI: fixes SPARK-802 2013-07-15 13:47:38 -07:00
Karen Feng b2aaa1199e Adds app name in HTML page titles on job web UI: fixes SPARK-806 2013-07-15 11:44:42 -07:00
Matei Zaharia d47c16f78d Add an option to disable reference tracking in Kryo 2013-07-15 01:55:54 +00:00
Matei Zaharia 10c05937bd Merge pull request #699 from pwendell/ui-env 
Add `Environment` tab to SparkUI.
 2013-07-14 11:45:18 -07:00
Patrick Wendell 4883586838 Responding to Matei's review 2013-07-14 10:37:26 -07:00
Matei Zaharia b91a218cea Cosmetic fixes to web UI 2013-07-14 07:31:33 +00:00
Matei Zaharia a44a7b1238 Determine Spark core classes better in getCallSite 2013-07-14 07:23:09 +00:00
root e271fde10b Fixed a delay scheduling bug in the YARN branch, found by Patrick 2013-07-14 06:24:29 +00:00
Patrick Wendell ddb97f0fdf Add Environment tab to SparkUI. 
This adds a tab which displays system property and classpath information. This
can be useful in debugging various types of issues such as:

1. Extra/incorrect Hadoop jars being included in the classpath
2. Spark launching with a different JRE version than intended
3. Spark system properties not being set to intended values
4. User added jars that conflict with Spark jars
 2013-07-13 16:14:40 -07:00
Matei Zaharia 77c69ae5a0 Merge pull request #697 from pwendell/block-locations 
Show block locations in Web UI.
 2013-07-12 23:05:21 -07:00
Matei Zaharia 5a7835c152 Merge pull request #691 from karenfeng/logpaging 
Create log pages
 2013-07-12 20:28:21 -07:00
Matei Zaharia 71ccca0cc1 Merge pull request #696 from woggle/executor-env 
Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath.sh
 2013-07-12 20:25:06 -07:00
Matei Zaharia 90fc3f30cd Merge pull request #692 from Reinvigorate/takeOrdered 
adding takeOrdered() to RDD
 2013-07-12 20:23:36 -07:00
Patrick Wendell 08150f19ab Minor style fix 2013-07-12 19:32:35 -07:00
Patrick Wendell 6855338e14 Show block locations in Web UI. 
This fixes SPARK-769. Support is added for enumerating the locations of blocks
in the UI. There is also some minor cleanup in StorageUtils.
 2013-07-12 19:30:32 -07:00
Charles Reiss 531a7e5574 Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath. 2013-07-12 12:58:25 -07:00
seanm a1662326e9 comment adjustment to takeOrdered 2013-07-12 08:38:19 -07:00
seanm a2c915fba8 giving order to top and making tests more clear 2013-07-11 18:55:00 -07:00
Karen Feng 5c67ca0278 Remove "Bytes" in lieu of String notation 2013-07-11 17:31:59 -07:00
Karen Feng 6d054487bf Replace default buffer value to 100 GB, changed buttons to use String notation, removed default buffer parameter in UI URLs 2013-07-11 17:12:17 -07:00
Karen Feng a32784109d Fixed links for "Back to Master" 2013-07-11 16:57:55 -07:00
Karen Feng ece2388585 Removed logPageLength from logPage 2013-07-11 16:35:56 -07:00
Karen Feng 9ed036ccdb Replaced logPageLength with byteLength to prevent buffer shrink bug 2013-07-11 16:33:53 -07:00
Karen Feng fdc226a14c Clarified start and end byte variable names 2013-07-11 15:36:43 -07:00
Karen Feng 5d5dbc39f6 getByteRange moved to WorkerWebUI, takes converted parameters, returns only start/end offset 2013-07-11 15:22:45 -07:00
Karen Feng 15fd11d657 Removed redundant calls to request by logPage 2013-07-11 15:01:50 -07:00
Karen Feng 11872888ca Created getByteRange function for logs and log pages, removed lastNBytes function 2013-07-11 14:56:37 -07:00
Matei Zaharia 018d04c64e Merge pull request #684 from woggle/mesos-classloader 
Explicitly set class loader for MesosSchedulerDriver callbacks.
 2013-07-11 12:48:37 -07:00