Kay Ousterhout
88049a214d
Fixed 3 bugs that caused UI to crash (including SPARK-810).
...
One bug caused the UI to crash if you try to look at a job's status
before any of the tasks have finished.
The second bug was a concurrency issue where two different threads
(the scheduling thread and a UI thread) could be reading/updating
the data structures in JobProgressListener concurrently.
The third bug mis-used an Option, also causing the UI to crash
under certain conditions.
2013-08-07 23:09:25 -07:00
Matei Zaharia
5133e4bebd
Merge pull request #790 from kayousterhout/fix_throughput
...
Fixed issue in UI that decreased scheduler throughput by 5x or more
2013-08-07 15:50:45 -07:00
Kay Ousterhout
b88e26248e
Fixed issue in UI that limited scheduler throughput.
...
Removal of items from ArrayBuffers in the UI code was slow and
significantly impacted scheduler throughput. This commit
improves scheduler throughput by 5x.
2013-08-07 14:42:05 -07:00
Matei Zaharia
3c8478e1fb
Merge pull request #747 from mateiz/improved-lr
...
Update the Python logistic regression example
2013-08-06 23:25:03 -07:00
Matei Zaharia
6b043a6f11
Merge pull request #724 from dlyubimov/SPARK-826
...
SPARK-826: fold(), reduce(), collect() always attempt to use java serialization
2013-08-06 22:31:02 -07:00
Matei Zaharia
7c4b7a53b1
Merge remote-tracking branch 'origin/pr/781'
...
Conflicts:
core/src/main/resources/spark/ui/static/webui.css
2013-08-06 17:19:49 -07:00
Matei Zaharia
de6c4c995a
Merge pull request #787 from ash211/master
...
Update spark-standalone.md
2013-08-06 17:09:50 -07:00
Karen Feng
908032e79b
Used saturated colors for progress bars
2013-08-06 16:52:21 -07:00
Andrew Ash
afc2c80fdb
Update spark-standalone.md
2013-08-07 00:44:43 +01:00
Karen Feng
8bc497fa10
Lightened color of progress bars
2013-08-06 16:33:05 -07:00
Karen Feng
ca1903ea63
Overlays progress text on top of bar
2013-08-06 15:45:42 -07:00
Matei Zaharia
df4d10d630
Merge pull request #779 from adatao/adatao-global-SparkEnv
...
[HOTFIX] Extend thread safety for SparkEnv.get()
2013-08-06 15:44:05 -07:00
Karen Feng
099528b6c4
Pre-sorts stage/env tables, changes text/link of stage summaries
2013-08-06 14:52:12 -07:00
Matei Zaharia
d2b0f0c23d
Merge pull request #770 from stayhf/SPARK-760-Java
...
Simple PageRank algorithm implementation in Java for SPARK-760
2013-08-06 14:49:39 -07:00
stayhf
882baee489
Got rid of unnecessary map function
2013-08-06 21:34:39 +00:00
Karen Feng
254a930730
Reverse sorts StageTable by submitted time
2013-08-06 14:18:38 -07:00
stayhf
326a7a82e0
changes as reviewer requested
2013-08-06 21:03:24 +00:00
Karen Feng
5ed5b73026
Sorts first column of env tables
2013-08-06 13:59:53 -07:00
Reynold Xin
d031f73679
Merge pull request #782 from WANdisco/master
...
SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD
2013-08-05 22:33:00 -07:00
Matei Zaharia
1b63dea816
Merge pull request #769 from markhamstra/NegativeCores
...
SPARK-847 + SPARK-845: Zombie workers and negative cores
2013-08-05 22:21:26 -07:00
Alexander Pivovarov
a30866438b
SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD
2013-08-05 21:48:43 -07:00
Matei Zaharia
828aff744d
Merge pull request #776 from gingsmith/master
...
adding matrix factorization data generator
2013-08-05 21:37:33 -07:00
Ginger Smith
bf7033f3eb
fixing formatting, style, and input
2013-08-05 21:26:24 -07:00
Matei Zaharia
8b277892c9
Merge pull request #774 from pwendell/job-description
...
Show user-defined job name in UI
2013-08-05 19:14:52 -07:00
Christopher Nguyen
b1bbbe699c
[HOTFIX] Mark lastSetSparkEnv @volatile in case it gets HotSpot-cached
...
On branch adatao-global-SparkEnv
Changes to be committed:
modified: core/src/main/scala/spark/SparkEnv.scala
2013-08-05 17:22:27 -07:00
Mark Hamstra
35d8f5ee52
Moved handling of timed out workers within the Master actor
2013-08-05 13:13:56 -07:00
Mark Hamstra
37ccf9301a
milliseconds -> seconds in timeOutDeadWorkers logging
2013-08-05 13:13:56 -07:00
Mark Hamstra
cdd1af562e
Timeout zombie workers
2013-08-05 13:13:56 -07:00
Mikhail Bautin
e8bec8365f
Only reduce the number of cores once when removing an executor
2013-08-05 13:13:56 -07:00
Karen Feng
95025afdec
Made most small fixes for SPARK-849 except for table sort, task progress overlay
2013-08-05 13:04:56 -07:00
Patrick Wendell
550b0cf48a
Merge pull request #780 from cybermaster/master
...
SPARK-850
2013-08-05 12:10:32 -07:00
Bill Zhao
33b9b155de
JBoss repository working now
2013-08-05 12:02:36 -07:00
Bill Zhao
9df66cd831
Merge branch 'master' of github.com:cybermaster/spark
2013-08-05 11:56:30 -07:00
Bill Zhao
87134b3648
SPARK-850: give better console message
2013-08-05 11:55:35 -07:00
Ginger Smith
8c8947e2b6
fixing formatting
2013-08-05 11:22:18 -07:00
Bill Zhao
d93d5fcaac
SPARK-850: Give better error message on the console
2013-08-05 10:09:03 -07:00
Christopher Nguyen
39e4fda76f
[HOTFIX] Extend thread safety for SparkEnv.get()
...
A ThreadLocal SparkEnv.env is facing various situations leading to
NullPointerExceptions, where SparkEnv.env set in one thread is not
gettable in another thread, but often assumed to be available.
See, e.g., https://groups.google.com/forum/#!topic/spark-developers/GLx8yunSj0A
This hotfixes SparkEnv.env to return either (a) the ThreadLocal
value if non-null, or (b) the previously set value in any thread.
This approach preserves SparkEnv.set() thread safety needed by
RDD.compute() and possibly other places. A refactoring that
parameterizes SparkEnv should be addressed subsequently.
On branch adatao-global-SparkEnv
Changes to be committed:
modified: core/src/main/scala/spark/SparkEnv.scala
2013-08-05 02:09:54 -07:00
stayhf
98fd62605d
Updated code with reviewer's suggestions
2013-08-05 00:30:28 +00:00
stayhf
a682637301
Simple PageRank algorithm implementation in Java for SPARK-760
2013-08-03 06:01:16 +00:00
Ginger Smith
4ab4df5edb
adding matrix factorization data generator
2013-08-02 22:22:36 -07:00
Matei Zaharia
22abbc10d6
Merge pull request #772 from karenfeng/ui-843
...
Show app duration
2013-08-02 16:37:59 -07:00
Matei Zaharia
abfa9e6f70
Increase Kryo buffer size in ALS since some arrays become big
2013-08-02 16:17:32 -07:00
Patrick Wendell
5b3784a79c
Show user-defined job name in UI
2013-08-02 15:47:41 -07:00
Karen Feng
b3ae5b25d5
Shows time the app has been running
2013-08-02 13:25:14 -07:00
Patrick Wendell
9d7dfd2d5a
Merge pull request #743 from pwendell/app-metrics
...
Add application metrics to standalone master
2013-08-01 17:41:58 -07:00
Patrick Wendell
6d7afd7ced
Merge pull request #768 from pwendell/pr-695
...
Minor clean-up of fair scheduler UI
2013-08-01 17:13:28 -07:00
Karen Feng
999eaac78c
Merge branch 'master' of https://github.com/mesos/spark
2013-08-01 15:54:56 -07:00
Matei Zaharia
e466a55a6b
Revert Mesos version to 0.9 since the 0.12 artifact has target Java 7
2013-08-01 15:45:21 -07:00
Patrick Wendell
d3c37ff120
Improving documentation in config file example
2013-08-01 15:26:26 -07:00
Patrick Wendell
f1d2ad550e
under_scores --> camelCase for config options
2013-08-01 15:26:26 -07:00