Commit graph

1552 commits

Author SHA1 Message Date
Karen Feng 2d6da9195a Alphabetized imports 2013-07-29 15:50:52 -07:00
Karen Feng 077f2dad22 Fixed outdated bugs 2013-07-27 16:39:36 -07:00
Karen Feng 5a93e3c58c Cleaned up code based on pwendell's suggestions 2013-07-27 15:55:26 -07:00
Karen Feng dcc4743a95 Moved val now to render 2013-07-27 12:52:53 -07:00
Karen Feng 1714693324 Current time called once with value now 2013-07-27 12:24:41 -07:00
Karen Feng bd4cc52e30 Made metrics Option instead of Some, fixed NullPointerException 2013-07-26 17:23:18 -07:00
Karen Feng 3fbe9eaac0 Displys shuffle read/write only if exists, wraps if statements, trims old vals, grabs current time once 2013-07-26 11:51:38 -07:00
Karen Feng 22faeab261 Split Shuffle Activity overview column for read/write 2013-07-25 17:14:18 -07:00
Karen Feng d4bbc8bd25 Shows totals for shuffle data and CPU time in Stage, homepage overviews including active time 2013-07-25 15:59:52 -07:00
Karen Feng 57009eef90 Fixed consistency of "success" status string 2013-07-24 13:43:09 -07:00
Karen Feng 4280e1768d Removed finished status for task info, changed name of success case 2013-07-24 12:48:48 -07:00
Karen Feng bd3931c874 Changed ifs with returns to if/else 2013-07-24 11:27:17 -07:00
Karen Feng 93c6015f82 Shows task status and running tasks on Stage Page: fixes SPARK-804 and 811 2013-07-24 10:53:02 -07:00
jerryshao 31ec72b243 Code refactor according to comments 2013-07-24 14:57:47 +08:00
jerryshao 8d1ef7f2df Code style changes 2013-07-24 14:57:47 +08:00
Andrew xia 05637de842 Change class xxxInstrumentation to class xxxSource 2013-07-24 14:57:47 +08:00
Andrew xia ed1a3bc206 continue to refactor code style and functions 2013-07-24 14:57:47 +08:00
jerryshao 5730193e0c Fix some typos 2013-07-24 14:57:47 +08:00
jerryshao a79f6077f0 Add Maven metrics library dependency and code changes 2013-07-24 14:57:47 +08:00
jerryshao 1daff54b2e Change Executor MetricsSystem initialize code to SparkEnv 2013-07-24 14:57:47 +08:00
Andrew xia 5f8802c1fb Register and init metricsSystem in SparkContext
Conflicts:

	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/SparkEnv.scala
2013-07-24 14:57:47 +08:00
Andrew xia 7d2eada451 Add metrics source of DAGScheduler and blockManager
Conflicts:

	core/src/main/scala/spark/SparkContext.scala
	core/src/main/scala/spark/SparkEnv.scala
2013-07-24 14:57:47 +08:00
jerryshao e9ac88754d Remove twice add Source bug and code clean 2013-07-24 14:57:47 +08:00
jerryshao 5ce5dc9fcd Add default properties to deal with no configure file situation 2013-07-24 14:57:47 +08:00
jerryshao 871bc1687e Add Executor instrumentation 2013-07-24 14:57:46 +08:00
jerryshao 7fb574bf66 Code clean and remarshal 2013-07-24 14:57:46 +08:00
Andrew xia 4d6dd67fa1 refactor metrics system
1.change source abstract class to support MetricRegistry
2.change master/work/jvm source class
2013-07-24 14:57:46 +08:00
jerryshao 03f9871116 MetricsSystem refactor 2013-07-24 14:57:46 +08:00
jerryshao c3daad3f65 Update metric source support for instrumentation 2013-07-24 14:57:46 +08:00
jerryshao 9dec8c73e6 Add Master and Worker instrumentation support 2013-07-24 14:57:46 +08:00
jerryshao 503acd3a37 Build metrics system framwork 2013-07-24 14:57:46 +08:00
Matei Zaharia b011329040 Merge pull request #727 from rxin/scheduler
Scheduler code style cleanup.
2013-07-23 22:50:09 -07:00
Matei Zaharia 876125b997 Merge pull request #726 from rxin/spark-826
SPARK-829: scheduler shouldn't hang if a task contains unserializable objects in its closure
2013-07-23 22:28:21 -07:00
Reynold Xin 3dae1df66f Moved non-serializable closure catching exception from submitStage to submitMissingTasks 2013-07-23 20:29:07 -07:00
Reynold Xin d33b8a2a0f Added comments on task closure serialization. 2013-07-23 20:28:39 -07:00
Reynold Xin 85ab8114bc Moved non-serializable closure catching exception from submitStage to submitMissingTasks 2013-07-23 20:25:58 -07:00
Matei Zaharia 6a31b7191d Small bug fix 2013-07-23 16:20:24 -07:00
Matei Zaharia 2f1736c396 Merge pull request #725 from karenfeng/task-start
Creates task start events
2013-07-23 15:53:30 -07:00
Karen Feng abc78cd331 Modifies instead of copies HashSets, fixes comment style 2013-07-23 15:47:16 -07:00
Karen Feng 383684daaa Replaces Seq with HashSet, removes redundant import 2013-07-23 15:33:27 -07:00
Reynold Xin f2422d4f29 SPARK-829: scheduler shouldn't hang if a task contains unserializable objects in its closure. 2013-07-23 15:30:20 -07:00
Reynold Xin 5ed38b4d1d Scheduler code style cleanup. 2013-07-23 15:28:59 -07:00
Reynold Xin 101b8cc78a SPARK-829: scheduler shouldn't hang if a task contains unserializable objects in its closure. 2013-07-23 15:28:20 -07:00
Karen Feng 9f2dbb2a7c Adds/removes active tasks only once 2013-07-23 15:10:09 -07:00
Karen Feng 0200801a55 Tracks task start events and shows number of active tasks on Executor UI 2013-07-23 13:35:43 -07:00
Matei Zaharia f369e0e51b Merge pull request #720 from ooyala/2013-07/persistent-rdds-api
Add a public method getCachedRdds to SparkContext
2013-07-23 13:22:27 -07:00
Evan Chan 4830e22562 Rename method per rxin feedback 2013-07-23 09:50:13 -07:00
Evan Chan 2c2bfbe294 Add toMap method to TimeStampedHashMap and use it 2013-07-23 01:36:44 -07:00
Matei Zaharia 401aac8b18 Merge pull request #719 from karenfeng/ui-808
Creates Executors tab for Jobs UI
2013-07-22 16:57:16 -07:00
Karen Feng 872c97ad82 Split task columns, memory columns sort by numeric value 2013-07-22 16:54:37 -07:00