ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3574 commits 6 branches 181 tags 397 MiB
Commit graph

3574 commits

This branch
This branch
All branches
Author SHA1 Message Date
Patrick Wendell 3d8f281604 Report compressed bytes read when calculating TaskMetrics 2013-08-11 16:25:57 -07:00
Matei Zaharia b09d4b79e8 Merge pull request #799 from woggle/sync-fix 
Remove extra synchronization in ResultTask
 2013-08-09 13:17:08 -07:00
Patrick Wendell cc6b92e80e Merge pull request #775 from pwendell/print-launch-command 
Log the launch command for Spark daemons
 2013-08-09 13:00:33 -07:00
Patrick Wendell 3970b580c2 Using quotes when printing out command 2013-08-09 11:53:32 -07:00
Charles Reiss 9dfc280f74 Remove extra synchronization in ResultTask 2013-08-09 11:09:02 -07:00
Matei Zaharia f94fc75c3f Merge pull request #788 from shane-huang/sparkjavaopts 
For standalone mode, add worker local env setting of SPARK_JAVA_OPTS as ...
 2013-08-09 10:04:03 -07:00
shane-huang 34c1f8d2da add comment in spark-env.sh.template for SPARK_JAVA_OPTS 
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
 2013-08-09 15:03:48 +08:00
Matei Zaharia 06303a62e5 Optimize JavaPageRank to use reduceByKey instead of groupByKey 2013-08-08 18:50:00 -07:00
Reynold Xin 01f20a941e Fixed a typo in mllib inline documentation. 2013-08-08 16:42:54 -07:00
Matei Zaharia 63b6e02bed Merge pull request #797 from mateiz/chill-0.3.1 
Update to Chill 0.3.1
 2013-08-08 14:02:02 -07:00
Matei Zaharia d1e1c1b24d Add test for Kryo with WrappedArray (which was failing in Chill 0.3.0) 2013-08-08 13:34:11 -07:00
Matei Zaharia 5a4003c1ac Update to Chill 0.3.1 2013-08-08 13:30:27 -07:00
Patrick Wendell 9955e5a4b7 Merge pull request #796 from pwendell/bootstrap-design 
Bootstrap re-design
 2013-08-08 11:03:38 -07:00
Patrick Wendell b4321edf68 Reverting boostrap change 2013-08-07 22:18:18 -07:00
Patrick Wendell 21392f2a73 Change I forgot to merge in 2013-08-07 21:45:32 -07:00
Patrick Wendell 706394b370 Bumping font size to 14px and fixing sytle issue in progress bars 2013-08-07 21:27:04 -07:00
Patrick Wendell 8c0d668468 Merge branch 'master' into bootstrap-design 
Conflicts:
	core/src/main/scala/spark/ui/UIUtils.scala
	core/src/main/scala/spark/ui/jobs/IndexPage.scala
	core/src/main/scala/spark/ui/storage/RDDPage.scala
 2013-08-07 21:06:03 -07:00
Matei Zaharia 5133e4bebd Merge pull request #790 from kayousterhout/fix_throughput 
Fixed issue in UI that decreased scheduler throughput by 5x or more
 2013-08-07 15:50:45 -07:00
Kay Ousterhout b88e26248e Fixed issue in UI that limited scheduler throughput. 
Removal of items from ArrayBuffers in the UI code was slow and
significantly impacted scheduler throughput. This commit
improves scheduler throughput by 5x.
 2013-08-07 14:42:05 -07:00
shane-huang cbc5107e36 For standalone mode, add worker local env setting of SPARK_JAVA_OPTS as default and let application env override default options if applicable 
Signed-off-by: shane-huang <shengsheng.huang@intel.com>
 2013-08-07 14:36:48 +08:00
Matei Zaharia 3c8478e1fb Merge pull request #747 from mateiz/improved-lr 
Update the Python logistic regression example
 2013-08-06 23:25:03 -07:00
Matei Zaharia 6b043a6f11 Merge pull request #724 from dlyubimov/SPARK-826 
SPARK-826: fold(), reduce(), collect() always attempt to use java serialization
 2013-08-06 22:31:02 -07:00
Matei Zaharia 7c4b7a53b1 Merge remote-tracking branch 'origin/pr/781' 
Conflicts:
	core/src/main/resources/spark/ui/static/webui.css
 2013-08-06 17:19:49 -07:00
Matei Zaharia de6c4c995a Merge pull request #787 from ash211/master 
Update spark-standalone.md
 2013-08-06 17:09:50 -07:00
Karen Feng 908032e79b Used saturated colors for progress bars 2013-08-06 16:52:21 -07:00
Andrew Ash afc2c80fdb Update spark-standalone.md 2013-08-07 00:44:43 +01:00
Karen Feng 8bc497fa10 Lightened color of progress bars 2013-08-06 16:33:05 -07:00
Karen Feng ca1903ea63 Overlays progress text on top of bar 2013-08-06 15:45:42 -07:00
Matei Zaharia df4d10d630 Merge pull request #779 from adatao/adatao-global-SparkEnv 
[HOTFIX] Extend thread safety for SparkEnv.get()
 2013-08-06 15:44:05 -07:00
Karen Feng 099528b6c4 Pre-sorts stage/env tables, changes text/link of stage summaries 2013-08-06 14:52:12 -07:00
Matei Zaharia d2b0f0c23d Merge pull request #770 from stayhf/SPARK-760-Java 
Simple PageRank algorithm implementation in Java for SPARK-760
 2013-08-06 14:49:39 -07:00
stayhf 882baee489 Got rid of unnecessary map function 2013-08-06 21:34:39 +00:00
Karen Feng 254a930730 Reverse sorts StageTable by submitted time 2013-08-06 14:18:38 -07:00
stayhf 326a7a82e0 changes as reviewer requested 2013-08-06 21:03:24 +00:00
Karen Feng 5ed5b73026 Sorts first column of env tables 2013-08-06 13:59:53 -07:00
Reynold Xin d031f73679 Merge pull request #782 from WANdisco/master 
SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD
 2013-08-05 22:33:00 -07:00
Matei Zaharia 1b63dea816 Merge pull request #769 from markhamstra/NegativeCores 
SPARK-847 + SPARK-845: Zombie workers and negative cores
 2013-08-05 22:21:26 -07:00
Alexander Pivovarov a30866438b SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD 2013-08-05 21:48:43 -07:00
Matei Zaharia 828aff744d Merge pull request #776 from gingsmith/master 
adding matrix factorization data generator
 2013-08-05 21:37:33 -07:00
Ginger Smith bf7033f3eb fixing formatting, style, and input 2013-08-05 21:26:24 -07:00
Matei Zaharia 8b277892c9 Merge pull request #774 from pwendell/job-description 
Show user-defined job name in UI
 2013-08-05 19:14:52 -07:00
Christopher Nguyen b1bbbe699c [HOTFIX] Mark lastSetSparkEnv @volatile in case it gets HotSpot-cached 
On branch adatao-global-SparkEnv
Changes to be committed:

	modified:   core/src/main/scala/spark/SparkEnv.scala
 2013-08-05 17:22:27 -07:00
Mark Hamstra 35d8f5ee52 Moved handling of timed out workers within the Master actor 2013-08-05 13:13:56 -07:00
Mark Hamstra 37ccf9301a milliseconds -> seconds in timeOutDeadWorkers logging 2013-08-05 13:13:56 -07:00
Mark Hamstra cdd1af562e Timeout zombie workers 2013-08-05 13:13:56 -07:00
Mikhail Bautin e8bec8365f Only reduce the number of cores once when removing an executor 2013-08-05 13:13:56 -07:00
Karen Feng 95025afdec Made most small fixes for SPARK-849 except for table sort, task progress overlay 2013-08-05 13:04:56 -07:00
Patrick Wendell 550b0cf48a Merge pull request #780 from cybermaster/master 
SPARK-850
 2013-08-05 12:10:32 -07:00
Bill Zhao 33b9b155de JBoss repository working now 2013-08-05 12:02:36 -07:00
Bill Zhao 9df66cd831 Merge branch 'master' of github.com:cybermaster/spark 2013-08-05 11:56:30 -07:00