ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3554 commits 6 branches 181 tags 397 MiB
Commit graph

1686 commits

Author SHA1 Message Date
Patrick Wendell 8c0d668468 Merge branch 'master' into bootstrap-design 
Conflicts:
	core/src/main/scala/spark/ui/UIUtils.scala
	core/src/main/scala/spark/ui/jobs/IndexPage.scala
	core/src/main/scala/spark/ui/storage/RDDPage.scala
 2013-08-07 21:06:03 -07:00
Kay Ousterhout b88e26248e Fixed issue in UI that limited scheduler throughput. 
Removal of items from ArrayBuffers in the UI code was slow and
significantly impacted scheduler throughput. This commit
improves scheduler throughput by 5x.
 2013-08-07 14:42:05 -07:00
Matei Zaharia 6b043a6f11 Merge pull request #724 from dlyubimov/SPARK-826 
SPARK-826: fold(), reduce(), collect() always attempt to use java serialization
 2013-08-06 22:31:02 -07:00
Matei Zaharia 7c4b7a53b1 Merge remote-tracking branch 'origin/pr/781' 
Conflicts:
	core/src/main/resources/spark/ui/static/webui.css
 2013-08-06 17:19:49 -07:00
Karen Feng 908032e79b Used saturated colors for progress bars 2013-08-06 16:52:21 -07:00
Karen Feng 8bc497fa10 Lightened color of progress bars 2013-08-06 16:33:05 -07:00
Karen Feng ca1903ea63 Overlays progress text on top of bar 2013-08-06 15:45:42 -07:00
Matei Zaharia df4d10d630 Merge pull request #779 from adatao/adatao-global-SparkEnv 
[HOTFIX] Extend thread safety for SparkEnv.get()
 2013-08-06 15:44:05 -07:00
Karen Feng 099528b6c4 Pre-sorts stage/env tables, changes text/link of stage summaries 2013-08-06 14:52:12 -07:00
Karen Feng 254a930730 Reverse sorts StageTable by submitted time 2013-08-06 14:18:38 -07:00
Karen Feng 5ed5b73026 Sorts first column of env tables 2013-08-06 13:59:53 -07:00
Reynold Xin d031f73679 Merge pull request #782 from WANdisco/master 
SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD
 2013-08-05 22:33:00 -07:00
Matei Zaharia 1b63dea816 Merge pull request #769 from markhamstra/NegativeCores 
SPARK-847 + SPARK-845: Zombie workers and negative cores
 2013-08-05 22:21:26 -07:00
Alexander Pivovarov a30866438b SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD 2013-08-05 21:48:43 -07:00
Matei Zaharia 8b277892c9 Merge pull request #774 from pwendell/job-description 
Show user-defined job name in UI
 2013-08-05 19:14:52 -07:00
Christopher Nguyen b1bbbe699c [HOTFIX] Mark lastSetSparkEnv @volatile in case it gets HotSpot-cached 
On branch adatao-global-SparkEnv
Changes to be committed:

	modified:   core/src/main/scala/spark/SparkEnv.scala
 2013-08-05 17:22:27 -07:00
Mark Hamstra 35d8f5ee52 Moved handling of timed out workers within the Master actor 2013-08-05 13:13:56 -07:00
Mark Hamstra 37ccf9301a milliseconds -> seconds in timeOutDeadWorkers logging 2013-08-05 13:13:56 -07:00
Mark Hamstra cdd1af562e Timeout zombie workers 2013-08-05 13:13:56 -07:00
Mikhail Bautin e8bec8365f Only reduce the number of cores once when removing an executor 2013-08-05 13:13:56 -07:00
Karen Feng 95025afdec Made most small fixes for SPARK-849 except for table sort, task progress overlay 2013-08-05 13:04:56 -07:00
Bill Zhao 87134b3648 SPARK-850: give better console message 2013-08-05 11:55:35 -07:00
Christopher Nguyen 39e4fda76f [HOTFIX] Extend thread safety for SparkEnv.get() 
A ThreadLocal SparkEnv.env is facing various situations leading to
NullPointerExceptions, where SparkEnv.env set in one thread is not
gettable in another thread, but often assumed to be available.

See, e.g., https://groups.google.com/forum/#!topic/spark-developers/GLx8yunSj0A

This hotfixes SparkEnv.env to return either (a) the ThreadLocal
value if non-null, or (b) the previously set value in any thread.

This approach preserves SparkEnv.set() thread safety needed by
RDD.compute() and possibly other places. A refactoring that
parameterizes SparkEnv should be addressed subsequently.

On branch adatao-global-SparkEnv
Changes to be committed:

	modified:   core/src/main/scala/spark/SparkEnv.scala
 2013-08-05 02:09:54 -07:00
Matei Zaharia 22abbc10d6 Merge pull request #772 from karenfeng/ui-843 
Show app duration
 2013-08-02 16:37:59 -07:00
Patrick Wendell 5b3784a79c Show user-defined job name in UI 2013-08-02 15:47:41 -07:00
Karen Feng b3ae5b25d5 Shows time the app has been running 2013-08-02 13:25:14 -07:00
Patrick Wendell 9d7dfd2d5a Merge pull request #743 from pwendell/app-metrics 
Add application metrics to standalone master
 2013-08-01 17:41:58 -07:00
Patrick Wendell f1d2ad550e under_scores --> camelCase for config options 2013-08-01 15:26:26 -07:00
Patrick Wendell 12d9c82c9b Small style fix 2013-08-01 15:25:52 -07:00
Patrick Wendell 37bc64a205 Adding application-level metrics. 
This adds metrics for applications in the deploy Master.
 2013-08-01 15:25:52 -07:00
Karen Feng 73692f3cb9 Unify, reduce body font size 2013-08-01 15:10:30 -07:00
Patrick Wendell 87fd321a5a Minor refactoring and code cleanup 2013-08-01 15:02:31 -07:00
Patrick Wendell b10199413a Slight refactoring to SparkContext functions 2013-08-01 15:00:42 -07:00
Patrick Wendell cfcd77b5da Increasing inter job arrival 2013-08-01 15:00:42 -07:00
Patrick Wendell 5faac7f4f3 Minor style fixes 2013-08-01 15:00:42 -07:00
Patrick Wendell 5e7b38fbb3 Merge pull request #695 from xiajunluan/pool_ui 
Enhance job ui in spark ui system with adding pool information
 2013-08-01 14:59:33 -07:00
Karen Feng 47600e9579 Removed hr margin 2013-08-01 14:57:04 -07:00
Karen Feng e648a62fc8 Inserted needed line break for log paging 2013-08-01 14:46:19 -07:00
Karen Feng 686d6266c4 Use nav pills instead of default 2013-08-01 14:41:49 -07:00
Karen Feng 86d372d17f Removed line breaks 2013-08-01 14:37:21 -07:00
Karen Feng 99803d88b9 Reduced all header sizes 2013-08-01 14:18:33 -07:00
Karen Feng d216d687ef Reduced size of table text to compact 2013-08-01 13:27:23 -07:00
Karen Feng 5dae283996 Merge branch 'master' of https://github.com/mesos/spark into bootstrap-update 2013-08-01 11:28:28 -07:00
Matei Zaharia 0a96493ac6 Merge pull request #760 from karenfeng/heading-update 
Clean up web UI page headers
 2013-08-01 11:27:17 -07:00
Patrick Wendell 9177bea2b4 Removing extra imports 2013-08-01 10:42:50 -07:00
Patrick Wendell 3e4d5e5f8b Merge branch 'master' into master-json 
Conflicts:
	core/src/main/scala/spark/deploy/master/ui/IndexPage.scala
 2013-08-01 10:42:07 -07:00
Patrick Wendell ffc034e4fb Import cleanup 2013-08-01 10:39:56 -07:00
Andrew xia d58502a156 fix bug of spark "SubmitStage" listener as unit test error 2013-08-01 23:21:41 +08:00
Andrew xia 3b5a11e765 change function name "setName" to "setProperties" as "setName" is also member of Thread class 2013-08-01 19:37:15 +08:00
Dmitriy Lyubimov cb6be5bd7e Merge remote-tracking branch 'mesos/master' into SPARK-826 
Conflicts:
	core/src/main/scala/spark/scheduler/cluster/ClusterTaskSetManager.scala
	core/src/main/scala/spark/scheduler/local/LocalTaskSetManager.scala
	core/src/test/scala/spark/KryoSerializerSuite.scala
 2013-07-31 22:09:22 -07:00