ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3682 commits 6 branches 181 tags 397 MiB
Commit graph

3682 commits

This branch
This branch
All branches
Author SHA1 Message Date
Mikhail Bautin e8bec8365f Only reduce the number of cores once when removing an executor 2013-08-05 13:13:56 -07:00
Karen Feng 95025afdec Made most small fixes for SPARK-849 except for table sort, task progress overlay 2013-08-05 13:04:56 -07:00
Patrick Wendell 550b0cf48a Merge pull request #780 from cybermaster/master 
SPARK-850
 2013-08-05 12:10:32 -07:00
Bill Zhao 33b9b155de JBoss repository working now 2013-08-05 12:02:36 -07:00
Bill Zhao 9df66cd831 Merge branch 'master' of github.com:cybermaster/spark 2013-08-05 11:56:30 -07:00
Bill Zhao 87134b3648 SPARK-850: give better console message 2013-08-05 11:55:35 -07:00
Ginger Smith 8c8947e2b6 fixing formatting 2013-08-05 11:22:18 -07:00
Bill Zhao d93d5fcaac SPARK-850: Give better error message on the console 2013-08-05 10:09:03 -07:00
Christopher Nguyen 39e4fda76f [HOTFIX] Extend thread safety for SparkEnv.get() 
A ThreadLocal SparkEnv.env is facing various situations leading to
NullPointerExceptions, where SparkEnv.env set in one thread is not
gettable in another thread, but often assumed to be available.

See, e.g., https://groups.google.com/forum/#!topic/spark-developers/GLx8yunSj0A

This hotfixes SparkEnv.env to return either (a) the ThreadLocal
value if non-null, or (b) the previously set value in any thread.

This approach preserves SparkEnv.set() thread safety needed by
RDD.compute() and possibly other places. A refactoring that
parameterizes SparkEnv should be addressed subsequently.

On branch adatao-global-SparkEnv
Changes to be committed:

	modified:   core/src/main/scala/spark/SparkEnv.scala
 2013-08-05 02:09:54 -07:00
stayhf 98fd62605d Updated code with reviewer's suggestions 2013-08-05 00:30:28 +00:00
Patrick Wendell f3660d5ab8 Make output formatting consistent between bash/scala 2013-08-03 21:30:15 -07:00
Shivaram Venkataraman 7388e27668 Move implicit arg to constructor for Java access. 2013-08-03 18:08:43 -07:00
Patrick Wendell ad94fbb322 Log the launch command for Spark executors 2013-08-03 09:19:46 -07:00
stayhf a682637301 Simple PageRank algorithm implementation in Java for SPARK-760 2013-08-03 06:01:16 +00:00
Ginger Smith 4ab4df5edb adding matrix factorization data generator 2013-08-02 22:22:36 -07:00
Shivaram Venkataraman 00339cc032 Refactor optimizers and create GLMs 
This change refactors the structure of GLMs to use mixins which maintain
a similar interface to other ML lib algorithms. This change also creates
an Optimizer trait which allows GLMs to be extended to use other optimization
techniques.
 2013-08-02 19:15:34 -07:00
Patrick Wendell b4905c383b Log the launch command for Spark daemons 
For debugging and analysis purposes, it's nice to have the exact command
used to launch Spark contained within the logs. This adds the necessary
hooks to make that possible.
 2013-08-02 16:58:19 -07:00
Matei Zaharia 22abbc10d6 Merge pull request #772 from karenfeng/ui-843 
Show app duration
 2013-08-02 16:37:59 -07:00
Matei Zaharia abfa9e6f70 Increase Kryo buffer size in ALS since some arrays become big 2013-08-02 16:17:32 -07:00
Patrick Wendell 5b3784a79c Show user-defined job name in UI 2013-08-02 15:47:41 -07:00
Karen Feng b3ae5b25d5 Shows time the app has been running 2013-08-02 13:25:14 -07:00
Patrick Wendell 9d7dfd2d5a Merge pull request #743 from pwendell/app-metrics 
Add application metrics to standalone master
 2013-08-01 17:41:58 -07:00
Patrick Wendell 6d7afd7ced Merge pull request #768 from pwendell/pr-695 
Minor clean-up of fair scheduler UI
 2013-08-01 17:13:28 -07:00
Karen Feng 999eaac78c Merge branch 'master' of https://github.com/mesos/spark 2013-08-01 15:54:56 -07:00
Matei Zaharia e466a55a6b Revert Mesos version to 0.9 since the 0.12 artifact has target Java 7 2013-08-01 15:45:21 -07:00
Patrick Wendell d3c37ff120 Improving documentation in config file example 2013-08-01 15:26:26 -07:00
Patrick Wendell f1d2ad550e under_scores --> camelCase for config options 2013-08-01 15:26:26 -07:00
Patrick Wendell 12d9c82c9b Small style fix 2013-08-01 15:25:52 -07:00
Patrick Wendell 37bc64a205 Adding application-level metrics. 
This adds metrics for applications in the deploy Master.
 2013-08-01 15:25:52 -07:00
Karen Feng 73692f3cb9 Unify, reduce body font size 2013-08-01 15:10:30 -07:00
Patrick Wendell 87fd321a5a Minor refactoring and code cleanup 2013-08-01 15:02:31 -07:00
Patrick Wendell b10199413a Slight refactoring to SparkContext functions 2013-08-01 15:00:42 -07:00
Patrick Wendell cfcd77b5da Increasing inter job arrival 2013-08-01 15:00:42 -07:00
Patrick Wendell 5faac7f4f3 Minor style fixes 2013-08-01 15:00:42 -07:00
Patrick Wendell 5e7b38fbb3 Merge pull request #695 from xiajunluan/pool_ui 
Enhance job ui in spark ui system with adding pool information
 2013-08-01 14:59:33 -07:00
Karen Feng 47600e9579 Removed hr margin 2013-08-01 14:57:04 -07:00
Karen Feng e648a62fc8 Inserted needed line break for log paging 2013-08-01 14:46:19 -07:00
Karen Feng 686d6266c4 Use nav pills instead of default 2013-08-01 14:41:49 -07:00
Karen Feng 86d372d17f Removed line breaks 2013-08-01 14:37:21 -07:00
Karen Feng 99803d88b9 Reduced all header sizes 2013-08-01 14:18:33 -07:00
Karen Feng d216d687ef Reduced size of table text to compact 2013-08-01 13:27:23 -07:00
Karen Feng 5dae283996 Merge branch 'master' of https://github.com/mesos/spark into bootstrap-update 2013-08-01 11:28:28 -07:00
Matei Zaharia 0a96493ac6 Merge pull request #760 from karenfeng/heading-update 
Clean up web UI page headers
 2013-08-01 11:27:17 -07:00
Patrick Wendell cb7dd86419 Merge pull request #758 from pwendell/master-json 
Add JSON path to master index page
 2013-08-01 11:06:10 -07:00
Patrick Wendell 9177bea2b4 Removing extra imports 2013-08-01 10:42:50 -07:00
Patrick Wendell 3e4d5e5f8b Merge branch 'master' into master-json 
Conflicts:
	core/src/main/scala/spark/deploy/master/ui/IndexPage.scala
 2013-08-01 10:42:07 -07:00
Patrick Wendell ffc034e4fb Import cleanup 2013-08-01 10:39:56 -07:00
Andrew xia d58502a156 fix bug of spark "SubmitStage" listener as unit test error 2013-08-01 23:21:41 +08:00
Andrew xia 3b5a11e765 change function name "setName" to "setProperties" as "setName" is also member of Thread class 2013-08-01 19:37:15 +08:00
Dmitriy Lyubimov d29ee3689b Merge fixes merge commit hasn't picked 2013-08-01 00:21:26 -07:00