ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3510 commits 6 branches 181 tags 397 MiB
Commit graph

3510 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia 7c4b7a53b1 Merge remote-tracking branch 'origin/pr/781' 
Conflicts:
	core/src/main/resources/spark/ui/static/webui.css
 2013-08-06 17:19:49 -07:00
Matei Zaharia de6c4c995a Merge pull request #787 from ash211/master 
Update spark-standalone.md
 2013-08-06 17:09:50 -07:00
Karen Feng 908032e79b Used saturated colors for progress bars 2013-08-06 16:52:21 -07:00
Andrew Ash afc2c80fdb Update spark-standalone.md 2013-08-07 00:44:43 +01:00
Karen Feng 8bc497fa10 Lightened color of progress bars 2013-08-06 16:33:05 -07:00
Karen Feng ca1903ea63 Overlays progress text on top of bar 2013-08-06 15:45:42 -07:00
Matei Zaharia df4d10d630 Merge pull request #779 from adatao/adatao-global-SparkEnv 
[HOTFIX] Extend thread safety for SparkEnv.get()
 2013-08-06 15:44:05 -07:00
Karen Feng 099528b6c4 Pre-sorts stage/env tables, changes text/link of stage summaries 2013-08-06 14:52:12 -07:00
Matei Zaharia d2b0f0c23d Merge pull request #770 from stayhf/SPARK-760-Java 
Simple PageRank algorithm implementation in Java for SPARK-760
 2013-08-06 14:49:39 -07:00
stayhf 882baee489 Got rid of unnecessary map function 2013-08-06 21:34:39 +00:00
Karen Feng 254a930730 Reverse sorts StageTable by submitted time 2013-08-06 14:18:38 -07:00
stayhf 326a7a82e0 changes as reviewer requested 2013-08-06 21:03:24 +00:00
Karen Feng 5ed5b73026 Sorts first column of env tables 2013-08-06 13:59:53 -07:00
Reynold Xin d031f73679 Merge pull request #782 from WANdisco/master 
SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD
 2013-08-05 22:33:00 -07:00
Matei Zaharia 1b63dea816 Merge pull request #769 from markhamstra/NegativeCores 
SPARK-847 + SPARK-845: Zombie workers and negative cores
 2013-08-05 22:21:26 -07:00
Alexander Pivovarov a30866438b SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD 2013-08-05 21:48:43 -07:00
Matei Zaharia 828aff744d Merge pull request #776 from gingsmith/master 
adding matrix factorization data generator
 2013-08-05 21:37:33 -07:00
Ginger Smith bf7033f3eb fixing formatting, style, and input 2013-08-05 21:26:24 -07:00
Matei Zaharia 8b277892c9 Merge pull request #774 from pwendell/job-description 
Show user-defined job name in UI
 2013-08-05 19:14:52 -07:00
Christopher Nguyen b1bbbe699c [HOTFIX] Mark lastSetSparkEnv @volatile in case it gets HotSpot-cached 
On branch adatao-global-SparkEnv
Changes to be committed:

	modified:   core/src/main/scala/spark/SparkEnv.scala
 2013-08-05 17:22:27 -07:00
Mark Hamstra 35d8f5ee52 Moved handling of timed out workers within the Master actor 2013-08-05 13:13:56 -07:00
Mark Hamstra 37ccf9301a milliseconds -> seconds in timeOutDeadWorkers logging 2013-08-05 13:13:56 -07:00
Mark Hamstra cdd1af562e Timeout zombie workers 2013-08-05 13:13:56 -07:00
Mikhail Bautin e8bec8365f Only reduce the number of cores once when removing an executor 2013-08-05 13:13:56 -07:00
Karen Feng 95025afdec Made most small fixes for SPARK-849 except for table sort, task progress overlay 2013-08-05 13:04:56 -07:00
Patrick Wendell 550b0cf48a Merge pull request #780 from cybermaster/master 
SPARK-850
 2013-08-05 12:10:32 -07:00
Bill Zhao 33b9b155de JBoss repository working now 2013-08-05 12:02:36 -07:00
Bill Zhao 9df66cd831 Merge branch 'master' of github.com:cybermaster/spark 2013-08-05 11:56:30 -07:00
Bill Zhao 87134b3648 SPARK-850: give better console message 2013-08-05 11:55:35 -07:00
Ginger Smith 8c8947e2b6 fixing formatting 2013-08-05 11:22:18 -07:00
Bill Zhao d93d5fcaac SPARK-850: Give better error message on the console 2013-08-05 10:09:03 -07:00
Christopher Nguyen 39e4fda76f [HOTFIX] Extend thread safety for SparkEnv.get() 
A ThreadLocal SparkEnv.env is facing various situations leading to
NullPointerExceptions, where SparkEnv.env set in one thread is not
gettable in another thread, but often assumed to be available.

See, e.g., https://groups.google.com/forum/#!topic/spark-developers/GLx8yunSj0A

This hotfixes SparkEnv.env to return either (a) the ThreadLocal
value if non-null, or (b) the previously set value in any thread.

This approach preserves SparkEnv.set() thread safety needed by
RDD.compute() and possibly other places. A refactoring that
parameterizes SparkEnv should be addressed subsequently.

On branch adatao-global-SparkEnv
Changes to be committed:

	modified:   core/src/main/scala/spark/SparkEnv.scala
 2013-08-05 02:09:54 -07:00
stayhf 98fd62605d Updated code with reviewer's suggestions 2013-08-05 00:30:28 +00:00
stayhf a682637301 Simple PageRank algorithm implementation in Java for SPARK-760 2013-08-03 06:01:16 +00:00
Ginger Smith 4ab4df5edb adding matrix factorization data generator 2013-08-02 22:22:36 -07:00
Matei Zaharia 22abbc10d6 Merge pull request #772 from karenfeng/ui-843 
Show app duration
 2013-08-02 16:37:59 -07:00
Matei Zaharia abfa9e6f70 Increase Kryo buffer size in ALS since some arrays become big 2013-08-02 16:17:32 -07:00
Patrick Wendell 5b3784a79c Show user-defined job name in UI 2013-08-02 15:47:41 -07:00
Karen Feng b3ae5b25d5 Shows time the app has been running 2013-08-02 13:25:14 -07:00
Patrick Wendell 9d7dfd2d5a Merge pull request #743 from pwendell/app-metrics 
Add application metrics to standalone master
 2013-08-01 17:41:58 -07:00
Patrick Wendell 6d7afd7ced Merge pull request #768 from pwendell/pr-695 
Minor clean-up of fair scheduler UI
 2013-08-01 17:13:28 -07:00
Karen Feng 999eaac78c Merge branch 'master' of https://github.com/mesos/spark 2013-08-01 15:54:56 -07:00
Matei Zaharia e466a55a6b Revert Mesos version to 0.9 since the 0.12 artifact has target Java 7 2013-08-01 15:45:21 -07:00
Patrick Wendell d3c37ff120 Improving documentation in config file example 2013-08-01 15:26:26 -07:00
Patrick Wendell f1d2ad550e under_scores --> camelCase for config options 2013-08-01 15:26:26 -07:00
Patrick Wendell 12d9c82c9b Small style fix 2013-08-01 15:25:52 -07:00
Patrick Wendell 37bc64a205 Adding application-level metrics. 
This adds metrics for applications in the deploy Master.
 2013-08-01 15:25:52 -07:00
Patrick Wendell 87fd321a5a Minor refactoring and code cleanup 2013-08-01 15:02:31 -07:00
Patrick Wendell b10199413a Slight refactoring to SparkContext functions 2013-08-01 15:00:42 -07:00
Patrick Wendell cfcd77b5da Increasing inter job arrival 2013-08-01 15:00:42 -07:00