ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3205 commits 6 branches 181 tags 397 MiB
Commit graph

3205 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia 4698a0d688 Shuffle ratings in a more efficient way at start of ALS 2013-07-15 02:54:11 +00:00
Matei Zaharia d47c16f78d Add an option to disable reference tracking in Kryo 2013-07-15 01:55:54 +00:00
Matei Zaharia 238d0e6893 Merge branch 'master' of github.com:mesos/spark 2013-07-15 00:30:32 +00:00
Matei Zaharia ed7fd501cf Make number of blocks in ALS configurable and lower the default 2013-07-15 00:30:10 +00:00
Matei Zaharia c7877d5e16 Merge pull request #689 from BlackNiuza/application_status 
Bug fix: SPARK-796
 2013-07-14 12:58:13 -07:00
Matei Zaharia 10c05937bd Merge pull request #699 from pwendell/ui-env 
Add `Environment` tab to SparkUI.
 2013-07-14 11:45:18 -07:00
Patrick Wendell 4883586838 Responding to Matei's review 2013-07-14 10:37:26 -07:00
BlackNiuza 00556a94c9 add spaces before curly braces and after for if conditions 2013-07-14 17:04:53 +08:00
Matei Zaharia 931e4c96ef Fix a comment 2013-07-14 08:03:13 +00:00
Matei Zaharia c5c38d1987 Some optimizations to loading phase of ALS 2013-07-14 07:59:50 +00:00
Matei Zaharia b91a218cea Cosmetic fixes to web UI 2013-07-14 07:31:33 +00:00
Matei Zaharia a44a7b1238 Determine Spark core classes better in getCallSite 2013-07-14 07:23:09 +00:00
root e271fde10b Fixed a delay scheduling bug in the YARN branch, found by Patrick 2013-07-14 06:24:29 +00:00
Matei Zaharia 668b0dc6a7 Merge branch 'master' of github.com:mesos/spark 2013-07-13 19:10:46 -07:00
Matei Zaharia cd28d9c147 Merge remote-tracking branch 'origin/pr/662' 
Conflicts:
	bin/compute-classpath.sh
 2013-07-13 19:10:00 -07:00
Patrick Wendell ddb97f0fdf Add Environment tab to SparkUI. 
This adds a tab which displays system property and classpath information. This
can be useful in debugging various types of issues such as:

1. Extra/incorrect Hadoop jars being included in the classpath
2. Spark launching with a different JRE version than intended
3. Spark system properties not being set to intended values
4. User added jars that conflict with Spark jars
 2013-07-13 16:14:40 -07:00
Matei Zaharia 89e8549f53 Merge pull request #698 from Reinvigorate/sm-deps-change 
changing com.google.code.findbugs maven coordinates
 2013-07-13 16:11:08 -07:00
Matei Zaharia 1889f9f450 Increase PermGen size 2013-07-13 14:43:21 -07:00
seanm c4d5b01e44 changing com.google.code.findbugs maven coordinates 2013-07-13 14:56:23 -06:00
Matei Zaharia 77c69ae5a0 Merge pull request #697 from pwendell/block-locations 
Show block locations in Web UI.
 2013-07-12 23:05:21 -07:00
Matei Zaharia 5a7835c152 Merge pull request #691 from karenfeng/logpaging 
Create log pages
 2013-07-12 20:28:21 -07:00
Matei Zaharia 71ccca0cc1 Merge pull request #696 from woggle/executor-env 
Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath.sh
 2013-07-12 20:25:06 -07:00
Matei Zaharia 90fc3f30cd Merge pull request #692 from Reinvigorate/takeOrdered 
adding takeOrdered() to RDD
 2013-07-12 20:23:36 -07:00
Patrick Wendell 08150f19ab Minor style fix 2013-07-12 19:32:35 -07:00
Patrick Wendell 6855338e14 Show block locations in Web UI. 
This fixes SPARK-769. Support is added for enumerating the locations of blocks
in the UI. There is also some minor cleanup in StorageUtils.
 2013-07-12 19:30:32 -07:00
Karen Feng 73984b96a8 Removed unit test of nonexistent function Utils.lastNBytes 2013-07-12 14:26:56 -07:00
Charles Reiss 531a7e5574 Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath. 2013-07-12 12:58:25 -07:00
Andy Konwinski cd7259b4b8 Fixes typos in Spark Streaming Programming Guide 
These typos were reported on the spark-users mailing list, see: https://groups.google.com/d/msg/spark-users/SyLGgJlKCrI/LpeBypOkSMUJ
 2013-07-12 11:51:14 -07:00
seanm a1662326e9 comment adjustment to takeOrdered 2013-07-12 08:38:19 -07:00
seanm a2c915fba8 giving order to top and making tests more clear 2013-07-11 18:55:00 -07:00
Karen Feng 5c67ca0278 Remove "Bytes" in lieu of String notation 2013-07-11 17:31:59 -07:00
Karen Feng 6d054487bf Replace default buffer value to 100 GB, changed buttons to use String notation, removed default buffer parameter in UI URLs 2013-07-11 17:12:17 -07:00
Karen Feng a32784109d Fixed links for "Back to Master" 2013-07-11 16:57:55 -07:00
Karen Feng ece2388585 Removed logPageLength from logPage 2013-07-11 16:35:56 -07:00
Karen Feng 9ed036ccdb Replaced logPageLength with byteLength to prevent buffer shrink bug 2013-07-11 16:33:53 -07:00
Karen Feng fdc226a14c Clarified start and end byte variable names 2013-07-11 15:36:43 -07:00
Karen Feng 5d5dbc39f6 getByteRange moved to WorkerWebUI, takes converted parameters, returns only start/end offset 2013-07-11 15:22:45 -07:00
Karen Feng 15fd11d657 Removed redundant calls to request by logPage 2013-07-11 15:01:50 -07:00
Karen Feng 11872888ca Created getByteRange function for logs and log pages, removed lastNBytes function 2013-07-11 14:56:37 -07:00
Matei Zaharia 018d04c64e Merge pull request #684 from woggle/mesos-classloader 
Explicitly set class loader for MesosSchedulerDriver callbacks.
 2013-07-11 12:48:37 -07:00
Karen Feng e3a3fcf61b Scrollbar on log pages appear automatically 2013-07-11 12:16:38 -07:00
Karen Feng 044d4577ec Fixed capitalization of log page 2013-07-11 12:02:15 -07:00
Karen Feng 0ecc33f0c8 Added byte range, page title with log name, previous/next bytes buttons, initialization to end of log, large default buffer, buggy back to master link 2013-07-11 11:25:58 -07:00
Matei Zaharia bc19477efb Merge pull request #693 from c0s/readme 
Updating README to reflect Scala 2.9.3 requirements
 2013-07-10 22:29:41 -07:00
Konstantin Boudnik ebf5b8c7c5 Updating README to reflect Scala 2.9.3 requirements 2013-07-10 22:16:06 -07:00
Karen Feng 74bd3fc680 Added byte range on log pages 2013-07-10 15:44:28 -07:00
Karen Feng 24196c91f0 Changed buffer to 10,000 bytes, created scrollbar for fixed-height log 2013-07-10 15:27:52 -07:00
Karen Feng f5f3b272f8 Fixed mixup of start/end, moved more import files 2013-07-10 14:52:29 -07:00
Karen Feng dbe948d9a2 Moved appropriate import files from UISuite to UtilsSuite 2013-07-10 14:15:41 -07:00
Karen Feng 5f8a20b4a8 Moved unit tests for Utils from UISuite to UtilsSuite 2013-07-10 13:53:39 -07:00