Reynold Xin
|
105f4d22e9
|
Removed Cache and SoftReferenceCache since they are no longer used.
|
2013-07-29 17:30:38 -07:00 |
|
Reynold Xin
|
17e62113d4
|
Moved DeployMessage's into its own DeployMessages object.
Also renamed MasterState to MasterStateResponse and WorkerState to WorkerStateResponse for clarity.
|
2013-07-29 17:14:44 -07:00 |
|
Karen Feng
|
87b821dc39
|
Fixed continuity of executorToTasksActive, changed color of progress bars
|
2013-07-29 16:50:51 -07:00 |
|
Karen Feng
|
c7b2788948
|
Merge branch 'master' of https://github.com/mesos/spark into bootstrap-update
Conflicts:
core/src/main/scala/spark/ui/jobs/IndexPage.scala
|
2013-07-29 16:36:07 -07:00 |
|
Patrick Wendell
|
c99b674405
|
Merge pull request #735 from karenfeng/ui-807
Totals for shuffle data and CPU time
|
2013-07-29 16:32:55 -07:00 |
|
Karen Feng
|
2d6da9195a
|
Alphabetized imports
|
2013-07-29 15:50:52 -07:00 |
|
Karen Feng
|
478a2886d9
|
Added started tasks to progress bar
|
2013-07-29 14:51:07 -07:00 |
|
Karen Feng
|
e04a37a332
|
Merge branch 'master' of https://github.com/mesos/spark into bootstrap-update
cially if it merges an updated upstream into a topic branch.
|
2013-07-29 14:32:48 -07:00 |
|
Reynold Xin
|
fe7298b587
|
Merge pull request #741 from pwendell/usability
Fix two small usability issues
|
2013-07-29 14:01:00 -07:00 |
|
Karen Feng
|
43a2cc15c0
|
Use Bootstrap progress bars in web UI
|
2013-07-29 13:37:24 -07:00 |
|
Matei Zaharia
|
b9d6783f36
|
Optimize Python take() to not compute entire first partition
|
2013-07-29 02:51:43 -04:00 |
|
Dmitriy Lyubimov
|
f5067abe85
|
changes per comments.
|
2013-07-27 23:08:00 -07:00 |
|
Karen Feng
|
077f2dad22
|
Fixed outdated bugs
|
2013-07-27 16:39:36 -07:00 |
|
Patrick Wendell
|
bcafb36c1e
|
Slight wording change
|
2013-07-27 16:03:50 -07:00 |
|
Patrick Wendell
|
8177165ac4
|
Log executor on finish
|
2013-07-27 16:02:06 -07:00 |
|
Patrick Wendell
|
c2223e6801
|
Improve catch scope and logging for client stop()
This does two things:
1. Catches the more general `TimeoutException`, since those can be thrown.
2. Logs at info level when a timeout is detected.
|
2013-07-27 16:02:06 -07:00 |
|
Karen Feng
|
5a93e3c58c
|
Cleaned up code based on pwendell's suggestions
|
2013-07-27 15:55:26 -07:00 |
|
Karen Feng
|
dcc4743a95
|
Moved val now to render
|
2013-07-27 12:52:53 -07:00 |
|
Karen Feng
|
1714693324
|
Current time called once with value now
|
2013-07-27 12:24:41 -07:00 |
|
Dmitriy Lyubimov
|
6a47cee721
|
style
|
2013-07-26 22:35:13 -07:00 |
|
Dmitriy Lyubimov
|
0c391feb73
|
Maximum task failures configurable
|
2013-07-26 22:34:43 -07:00 |
|
Karen Feng
|
bd4cc52e30
|
Made metrics Option instead of Some, fixed NullPointerException
|
2013-07-26 17:23:18 -07:00 |
|
Reynold Xin
|
cb366774c8
|
Merge pull request #738 from harsha2010/pruning
Fix bug in Partition Pruning.
|
2013-07-26 16:59:30 -07:00 |
|
harshars
|
392d7474fd
|
Code review
|
2013-07-26 15:23:15 -07:00 |
|
harshars
|
72cf7ec0e5
|
Indentation
|
2013-07-26 15:16:41 -07:00 |
|
harshars
|
822aac8f5a
|
Indentation
|
2013-07-26 15:10:32 -07:00 |
|
harshars
|
743fc4e7aa
|
Fix Bug in Partition Pruning, index of Pruned Partitions should inherit from parent
|
2013-07-26 14:35:17 -07:00 |
|
Karen Feng
|
3fbe9eaac0
|
Displys shuffle read/write only if exists, wraps if statements, trims old vals, grabs current time once
|
2013-07-26 11:51:38 -07:00 |
|
Karen Feng
|
22faeab261
|
Split Shuffle Activity overview column for read/write
|
2013-07-25 17:14:18 -07:00 |
|
Karen Feng
|
d4bbc8bd25
|
Shows totals for shuffle data and CPU time in Stage, homepage overviews including active time
|
2013-07-25 15:59:52 -07:00 |
|
Charles Reiss
|
a6de90c927
|
For standalone mode, get JAVA_HOME, SPARK_JAVA_OPTS, SPARK_LIBRARY_PATH from application env, not worker env
|
2013-07-25 12:42:30 -07:00 |
|
ryanlecompte
|
e56aa75de0
|
fix wrapping
|
2013-07-24 22:08:09 -07:00 |
|
ryanlecompte
|
8e0939f5a9
|
refactor Kryo serializer support to use chill/chill-java
|
2013-07-24 20:43:57 -07:00 |
|
Karen Feng
|
57009eef90
|
Fixed consistency of "success" status string
|
2013-07-24 13:43:09 -07:00 |
|
Karen Feng
|
4280e1768d
|
Removed finished status for task info, changed name of success case
|
2013-07-24 12:48:48 -07:00 |
|
Karen Feng
|
bd3931c874
|
Changed ifs with returns to if/else
|
2013-07-24 11:27:17 -07:00 |
|
Karen Feng
|
93c6015f82
|
Shows task status and running tasks on Stage Page: fixes SPARK-804 and 811
|
2013-07-24 10:53:02 -07:00 |
|
jerryshao
|
31ec72b243
|
Code refactor according to comments
|
2013-07-24 14:57:47 +08:00 |
|
jerryshao
|
8d1ef7f2df
|
Code style changes
|
2013-07-24 14:57:47 +08:00 |
|
Andrew xia
|
05637de842
|
Change class xxxInstrumentation to class xxxSource
|
2013-07-24 14:57:47 +08:00 |
|
Andrew xia
|
ed1a3bc206
|
continue to refactor code style and functions
|
2013-07-24 14:57:47 +08:00 |
|
jerryshao
|
5730193e0c
|
Fix some typos
|
2013-07-24 14:57:47 +08:00 |
|
jerryshao
|
a79f6077f0
|
Add Maven metrics library dependency and code changes
|
2013-07-24 14:57:47 +08:00 |
|
jerryshao
|
1daff54b2e
|
Change Executor MetricsSystem initialize code to SparkEnv
|
2013-07-24 14:57:47 +08:00 |
|
Andrew xia
|
5f8802c1fb
|
Register and init metricsSystem in SparkContext
Conflicts:
core/src/main/scala/spark/SparkContext.scala
core/src/main/scala/spark/SparkEnv.scala
|
2013-07-24 14:57:47 +08:00 |
|
Andrew xia
|
7d2eada451
|
Add metrics source of DAGScheduler and blockManager
Conflicts:
core/src/main/scala/spark/SparkContext.scala
core/src/main/scala/spark/SparkEnv.scala
|
2013-07-24 14:57:47 +08:00 |
|
jerryshao
|
e9ac88754d
|
Remove twice add Source bug and code clean
|
2013-07-24 14:57:47 +08:00 |
|
jerryshao
|
5ce5dc9fcd
|
Add default properties to deal with no configure file situation
|
2013-07-24 14:57:47 +08:00 |
|
jerryshao
|
871bc1687e
|
Add Executor instrumentation
|
2013-07-24 14:57:46 +08:00 |
|
jerryshao
|
7fb574bf66
|
Code clean and remarshal
|
2013-07-24 14:57:46 +08:00 |
|
Andrew xia
|
4d6dd67fa1
|
refactor metrics system
1.change source abstract class to support MetricRegistry
2.change master/work/jvm source class
|
2013-07-24 14:57:46 +08:00 |
|
jerryshao
|
03f9871116
|
MetricsSystem refactor
|
2013-07-24 14:57:46 +08:00 |
|
jerryshao
|
c3daad3f65
|
Update metric source support for instrumentation
|
2013-07-24 14:57:46 +08:00 |
|
jerryshao
|
9dec8c73e6
|
Add Master and Worker instrumentation support
|
2013-07-24 14:57:46 +08:00 |
|
jerryshao
|
503acd3a37
|
Build metrics system framwork
|
2013-07-24 14:57:46 +08:00 |
|
Matei Zaharia
|
b011329040
|
Merge pull request #727 from rxin/scheduler
Scheduler code style cleanup.
|
2013-07-23 22:50:09 -07:00 |
|
Matei Zaharia
|
876125b997
|
Merge pull request #726 from rxin/spark-826
SPARK-829: scheduler shouldn't hang if a task contains unserializable objects in its closure
|
2013-07-23 22:28:21 -07:00 |
|
Reynold Xin
|
3dae1df66f
|
Moved non-serializable closure catching exception from submitStage to submitMissingTasks
|
2013-07-23 20:29:07 -07:00 |
|
Reynold Xin
|
d33b8a2a0f
|
Added comments on task closure serialization.
|
2013-07-23 20:28:39 -07:00 |
|
Reynold Xin
|
85ab8114bc
|
Moved non-serializable closure catching exception from submitStage to submitMissingTasks
|
2013-07-23 20:25:58 -07:00 |
|
Matei Zaharia
|
6a31b7191d
|
Small bug fix
|
2013-07-23 16:20:24 -07:00 |
|
Matei Zaharia
|
2f1736c396
|
Merge pull request #725 from karenfeng/task-start
Creates task start events
|
2013-07-23 15:53:30 -07:00 |
|
Karen Feng
|
abc78cd331
|
Modifies instead of copies HashSets, fixes comment style
|
2013-07-23 15:47:16 -07:00 |
|
Karen Feng
|
383684daaa
|
Replaces Seq with HashSet, removes redundant import
|
2013-07-23 15:33:27 -07:00 |
|
Reynold Xin
|
f2422d4f29
|
SPARK-829: scheduler shouldn't hang if a task contains unserializable objects in its closure.
|
2013-07-23 15:30:20 -07:00 |
|
Reynold Xin
|
5ed38b4d1d
|
Scheduler code style cleanup.
|
2013-07-23 15:28:59 -07:00 |
|
Reynold Xin
|
101b8cc78a
|
SPARK-829: scheduler shouldn't hang if a task contains unserializable objects in its closure.
|
2013-07-23 15:28:20 -07:00 |
|
Karen Feng
|
9f2dbb2a7c
|
Adds/removes active tasks only once
|
2013-07-23 15:10:09 -07:00 |
|
Karen Feng
|
0200801a55
|
Tracks task start events and shows number of active tasks on Executor UI
|
2013-07-23 13:35:43 -07:00 |
|
Matei Zaharia
|
f369e0e51b
|
Merge pull request #720 from ooyala/2013-07/persistent-rdds-api
Add a public method getCachedRdds to SparkContext
|
2013-07-23 13:22:27 -07:00 |
|
Evan Chan
|
4830e22562
|
Rename method per rxin feedback
|
2013-07-23 09:50:13 -07:00 |
|
Evan Chan
|
2c2bfbe294
|
Add toMap method to TimeStampedHashMap and use it
|
2013-07-23 01:36:44 -07:00 |
|
Matei Zaharia
|
401aac8b18
|
Merge pull request #719 from karenfeng/ui-808
Creates Executors tab for Jobs UI
|
2013-07-22 16:57:16 -07:00 |
|
Karen Feng
|
872c97ad82
|
Split task columns, memory columns sort by numeric value
|
2013-07-22 16:54:37 -07:00 |
|
Karen Feng
|
2eea974795
|
Executors UI now calls executor ID from TaskInfo instead of TaskMetrics
|
2013-07-22 15:15:54 -07:00 |
|
Karen Feng
|
85c4d7bf3b
|
Shows number of complete/total/failed tasks (bug: failed tasks assigned to null executor)
|
2013-07-22 14:35:47 -07:00 |
|
Josh Rosen
|
f649dabb4a
|
Fix bug: DoubleRDDFunctions.sampleStdev() computed non-sample stdev().
Update JavaDoubleRDD to add new methods and docs.
Fixes SPARK-825.
|
2013-07-22 13:21:48 -07:00 |
|
Karen Feng
|
8901f379c9
|
Fixed memory used/remaining/total bug
|
2013-07-22 09:58:03 -07:00 |
|
Karen Feng
|
636b19f833
|
Merge branch 'master' of https://github.com/mesos/spark into ui-808
|
2013-07-22 09:53:26 -07:00 |
|
Evan Chan
|
0337d88321
|
Add a public method getCachedRdds to SparkContext
|
2013-07-21 18:26:14 -07:00 |
|
Karen Feng
|
865dc63bac
|
Changed table format for executors
|
2013-07-19 15:57:01 -07:00 |
|
Karen Feng
|
81bb5dc640
|
Creates Executors tab for application with RDD block and memory/disk used, solves SPARK-808
|
2013-07-19 14:08:30 -07:00 |
|
Konstantin Boudnik
|
cfce9a6a36
|
Regression: default webui-port can't be set via command line "--webui-port" anymore
|
2013-07-19 14:00:58 -07:00 |
|
Liang-Chi Hsieh
|
aa6f83289b
|
A better fix for giving local jars unde Yarn mode.
|
2013-07-19 22:25:28 +08:00 |
|
Liang-Chi Hsieh
|
a613628c50
|
Do not copy local jars given to SparkContext in yarn mode since the Context is not running on local. This bug causes failure when jars can not be found. Example codes (such as spark.examples.SparkPi) can not work without this fix under yarn mode.
|
2013-07-19 16:59:12 +08:00 |
|
Matei Zaharia
|
af3c9d5042
|
Add Apache license headers and LICENSE and NOTICE files
|
2013-07-16 17:21:33 -07:00 |
|
Matei Zaharia
|
b1f9f64743
|
Merge branch 'master' of github.com:mesos/spark
|
2013-07-16 11:01:53 -07:00 |
|
Matei Zaharia
|
5c388808a8
|
SPARK-814: Result stages should be named after action
|
2013-07-16 11:01:14 -07:00 |
|
Reynold Xin
|
69316603d6
|
Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions.
|
2013-07-15 22:50:11 -07:00 |
|
Karen Feng
|
6dc7c9bfb1
|
Removed job UI column, linked description to job UI
|
2013-07-15 16:33:50 -07:00 |
|
Karen Feng
|
fbf5aa761e
|
Removed log message, added field in master UI to link to log UI
|
2013-07-15 15:50:03 -07:00 |
|
Karen Feng
|
eac381a957
|
Merge branch 'ui-802' of https://github.com/karenfeng/spark into ui-802
|
2013-07-15 15:48:44 -07:00 |
|
Karen Feng
|
3955711250
|
Added field to master UI with link to job UI
|
2013-07-15 15:47:21 -07:00 |
|
Karen Feng
|
0d78b6d9cd
|
Links to job UI from standalone deploy cluster web UI: fixes SPARK-802
|
2013-07-15 13:47:38 -07:00 |
|
Karen Feng
|
b2aaa1199e
|
Adds app name in HTML page titles on job web UI: fixes SPARK-806
|
2013-07-15 11:44:42 -07:00 |
|
Matei Zaharia
|
d47c16f78d
|
Add an option to disable reference tracking in Kryo
|
2013-07-15 01:55:54 +00:00 |
|
Matei Zaharia
|
10c05937bd
|
Merge pull request #699 from pwendell/ui-env
Add `Environment` tab to SparkUI.
|
2013-07-14 11:45:18 -07:00 |
|
Patrick Wendell
|
4883586838
|
Responding to Matei's review
|
2013-07-14 10:37:26 -07:00 |
|
Matei Zaharia
|
b91a218cea
|
Cosmetic fixes to web UI
|
2013-07-14 07:31:33 +00:00 |
|
Matei Zaharia
|
a44a7b1238
|
Determine Spark core classes better in getCallSite
|
2013-07-14 07:23:09 +00:00 |
|