ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3292 commits 6 branches 181 tags 397 MiB
Commit graph

3292 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia cd28d9c147 Merge remote-tracking branch 'origin/pr/662' 
Conflicts:
	bin/compute-classpath.sh
 2013-07-13 19:10:00 -07:00
Patrick Wendell ddb97f0fdf Add Environment tab to SparkUI. 
This adds a tab which displays system property and classpath information. This
can be useful in debugging various types of issues such as:

1. Extra/incorrect Hadoop jars being included in the classpath
2. Spark launching with a different JRE version than intended
3. Spark system properties not being set to intended values
4. User added jars that conflict with Spark jars
 2013-07-13 16:14:40 -07:00
Matei Zaharia 89e8549f53 Merge pull request #698 from Reinvigorate/sm-deps-change 
changing com.google.code.findbugs maven coordinates
 2013-07-13 16:11:08 -07:00
Matei Zaharia 1889f9f450 Increase PermGen size 2013-07-13 14:43:21 -07:00
seanm c4d5b01e44 changing com.google.code.findbugs maven coordinates 2013-07-13 14:56:23 -06:00
Matei Zaharia 77c69ae5a0 Merge pull request #697 from pwendell/block-locations 
Show block locations in Web UI.
 2013-07-12 23:05:21 -07:00
Matei Zaharia 5a7835c152 Merge pull request #691 from karenfeng/logpaging 
Create log pages
 2013-07-12 20:28:21 -07:00
Matei Zaharia 71ccca0cc1 Merge pull request #696 from woggle/executor-env 
Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath.sh
 2013-07-12 20:25:06 -07:00
Matei Zaharia 90fc3f30cd Merge pull request #692 from Reinvigorate/takeOrdered 
adding takeOrdered() to RDD
 2013-07-12 20:23:36 -07:00
Patrick Wendell 08150f19ab Minor style fix 2013-07-12 19:32:35 -07:00
Patrick Wendell 6855338e14 Show block locations in Web UI. 
This fixes SPARK-769. Support is added for enumerating the locations of blocks
in the UI. There is also some minor cleanup in StorageUtils.
 2013-07-12 19:30:32 -07:00
Karen Feng 73984b96a8 Removed unit test of nonexistent function Utils.lastNBytes 2013-07-12 14:26:56 -07:00
Charles Reiss 531a7e5574 Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath. 2013-07-12 12:58:25 -07:00
Andy Konwinski cd7259b4b8 Fixes typos in Spark Streaming Programming Guide 
These typos were reported on the spark-users mailing list, see: https://groups.google.com/d/msg/spark-users/SyLGgJlKCrI/LpeBypOkSMUJ
 2013-07-12 11:51:14 -07:00
seanm a1662326e9 comment adjustment to takeOrdered 2013-07-12 08:38:19 -07:00
seanm a2c915fba8 giving order to top and making tests more clear 2013-07-11 18:55:00 -07:00
Karen Feng 5c67ca0278 Remove "Bytes" in lieu of String notation 2013-07-11 17:31:59 -07:00
Karen Feng 6d054487bf Replace default buffer value to 100 GB, changed buttons to use String notation, removed default buffer parameter in UI URLs 2013-07-11 17:12:17 -07:00
Karen Feng a32784109d Fixed links for "Back to Master" 2013-07-11 16:57:55 -07:00
Karen Feng ece2388585 Removed logPageLength from logPage 2013-07-11 16:35:56 -07:00
Karen Feng 9ed036ccdb Replaced logPageLength with byteLength to prevent buffer shrink bug 2013-07-11 16:33:53 -07:00
Karen Feng fdc226a14c Clarified start and end byte variable names 2013-07-11 15:36:43 -07:00
Karen Feng 5d5dbc39f6 getByteRange moved to WorkerWebUI, takes converted parameters, returns only start/end offset 2013-07-11 15:22:45 -07:00
Karen Feng 15fd11d657 Removed redundant calls to request by logPage 2013-07-11 15:01:50 -07:00
Karen Feng 11872888ca Created getByteRange function for logs and log pages, removed lastNBytes function 2013-07-11 14:56:37 -07:00
Matei Zaharia 018d04c64e Merge pull request #684 from woggle/mesos-classloader 
Explicitly set class loader for MesosSchedulerDriver callbacks.
 2013-07-11 12:48:37 -07:00
Karen Feng e3a3fcf61b Scrollbar on log pages appear automatically 2013-07-11 12:16:38 -07:00
Karen Feng 044d4577ec Fixed capitalization of log page 2013-07-11 12:02:15 -07:00
Karen Feng 0ecc33f0c8 Added byte range, page title with log name, previous/next bytes buttons, initialization to end of log, large default buffer, buggy back to master link 2013-07-11 11:25:58 -07:00
Matei Zaharia bc19477efb Merge pull request #693 from c0s/readme 
Updating README to reflect Scala 2.9.3 requirements
 2013-07-10 22:29:41 -07:00
Konstantin Boudnik ebf5b8c7c5 Updating README to reflect Scala 2.9.3 requirements 2013-07-10 22:16:06 -07:00
Karen Feng 74bd3fc680 Added byte range on log pages 2013-07-10 15:44:28 -07:00
Karen Feng 24196c91f0 Changed buffer to 10,000 bytes, created scrollbar for fixed-height log 2013-07-10 15:27:52 -07:00
Karen Feng f5f3b272f8 Fixed mixup of start/end, moved more import files 2013-07-10 14:52:29 -07:00
Karen Feng dbe948d9a2 Moved appropriate import files from UISuite to UtilsSuite 2013-07-10 14:15:41 -07:00
Karen Feng 5f8a20b4a8 Moved unit tests for Utils from UISuite to UtilsSuite 2013-07-10 13:53:39 -07:00
Karen Feng 0d4580360b Fixed docstring of offsetBytes to match params and wrapped for 100+ character lines 2013-07-10 13:24:26 -07:00
Karen Feng 04263e4d46 Made some minor style changes 2013-07-10 13:15:42 -07:00
Karen Feng cfb6447ac4 Fixed for nonexistent bytes, added unit tests, changed stdout-page to stdout 2013-07-10 11:47:57 -07:00
seanm ee4ce2fc51 adding takeOrdered to java API 2013-07-10 10:46:04 -07:00
seanm 24705d0f46 adding takeOrdered() to RDD 2013-07-10 10:33:11 -07:00
Karen Feng 620a6974c6 Allows for larger files, refactors lastNBytes, removes old Log column, fixes imports, uses map 2013-07-10 10:20:53 -07:00
BlackNiuza ce18b50d5f set SUCCEEDED for all master in shutdown hook 2013-07-10 19:11:43 +08:00
Karen Feng b6072b58bf Fixes style, makes "std__-page" consistent, reads only parts of files 2013-07-09 17:25:10 -07:00
Karen Feng 13fc6f248c Clean commit of log paging 2013-07-09 14:17:15 -07:00
BlackNiuza aaa7b081df according to mridulm's comments to adjust the code 2013-07-09 20:03:01 +08:00
Charles Reiss e47253e0cc Reset ClassLoader in MesosSchedulerBackend, too. (per review comments). 
Also set ClassLoader for all mesos callbacks, not just statusUpdate,
registered.
 2013-07-09 01:23:23 -07:00
BlackNiuza c1d44be805 Bug fix: SPARK-796 2013-07-09 15:18:28 +08:00
Matei Zaharia 7dcda9ae74 Merge pull request #688 from markhamstra/scalaDependencies 
Fixed SPARK-795 with explicit dependencies
 2013-07-08 23:24:23 -07:00
Matei Zaharia 638927b78e Merge pull request #683 from shivaram/sbt-test-fix 
Remove some stack traces from sbt test output
 2013-07-08 22:58:50 -07:00