ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3210 commits 6 branches 181 tags 397 MiB
Commit graph

1490 commits

Author SHA1 Message Date
Karen Feng eac381a957 Merge branch 'ui-802' of https://github.com/karenfeng/spark into ui-802 2013-07-15 15:48:44 -07:00
Karen Feng 3955711250 Added field to master UI with link to job UI 2013-07-15 15:47:21 -07:00
Karen Feng 0d78b6d9cd Links to job UI from standalone deploy cluster web UI: fixes SPARK-802 2013-07-15 13:47:38 -07:00
Karen Feng b2aaa1199e Adds app name in HTML page titles on job web UI: fixes SPARK-806 2013-07-15 11:44:42 -07:00
Matei Zaharia d47c16f78d Add an option to disable reference tracking in Kryo 2013-07-15 01:55:54 +00:00
Matei Zaharia 10c05937bd Merge pull request #699 from pwendell/ui-env 
Add `Environment` tab to SparkUI.
 2013-07-14 11:45:18 -07:00
Patrick Wendell 4883586838 Responding to Matei's review 2013-07-14 10:37:26 -07:00
Matei Zaharia b91a218cea Cosmetic fixes to web UI 2013-07-14 07:31:33 +00:00
Matei Zaharia a44a7b1238 Determine Spark core classes better in getCallSite 2013-07-14 07:23:09 +00:00
root e271fde10b Fixed a delay scheduling bug in the YARN branch, found by Patrick 2013-07-14 06:24:29 +00:00
Patrick Wendell ddb97f0fdf Add Environment tab to SparkUI. 
This adds a tab which displays system property and classpath information. This
can be useful in debugging various types of issues such as:

1. Extra/incorrect Hadoop jars being included in the classpath
2. Spark launching with a different JRE version than intended
3. Spark system properties not being set to intended values
4. User added jars that conflict with Spark jars
 2013-07-13 16:14:40 -07:00
Matei Zaharia 77c69ae5a0 Merge pull request #697 from pwendell/block-locations 
Show block locations in Web UI.
 2013-07-12 23:05:21 -07:00
Matei Zaharia 5a7835c152 Merge pull request #691 from karenfeng/logpaging 
Create log pages
 2013-07-12 20:28:21 -07:00
Matei Zaharia 71ccca0cc1 Merge pull request #696 from woggle/executor-env 
Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath.sh
 2013-07-12 20:25:06 -07:00
Matei Zaharia 90fc3f30cd Merge pull request #692 from Reinvigorate/takeOrdered 
adding takeOrdered() to RDD
 2013-07-12 20:23:36 -07:00
Patrick Wendell 08150f19ab Minor style fix 2013-07-12 19:32:35 -07:00
Patrick Wendell 6855338e14 Show block locations in Web UI. 
This fixes SPARK-769. Support is added for enumerating the locations of blocks
in the UI. There is also some minor cleanup in StorageUtils.
 2013-07-12 19:30:32 -07:00
Charles Reiss 531a7e5574 Pass executor env vars (e.g. SPARK_CLASSPATH) to compute-classpath. 2013-07-12 12:58:25 -07:00
seanm a1662326e9 comment adjustment to takeOrdered 2013-07-12 08:38:19 -07:00
seanm a2c915fba8 giving order to top and making tests more clear 2013-07-11 18:55:00 -07:00
Karen Feng 5c67ca0278 Remove "Bytes" in lieu of String notation 2013-07-11 17:31:59 -07:00
Karen Feng 6d054487bf Replace default buffer value to 100 GB, changed buttons to use String notation, removed default buffer parameter in UI URLs 2013-07-11 17:12:17 -07:00
Karen Feng a32784109d Fixed links for "Back to Master" 2013-07-11 16:57:55 -07:00
Karen Feng ece2388585 Removed logPageLength from logPage 2013-07-11 16:35:56 -07:00
Karen Feng 9ed036ccdb Replaced logPageLength with byteLength to prevent buffer shrink bug 2013-07-11 16:33:53 -07:00
Karen Feng fdc226a14c Clarified start and end byte variable names 2013-07-11 15:36:43 -07:00
Karen Feng 5d5dbc39f6 getByteRange moved to WorkerWebUI, takes converted parameters, returns only start/end offset 2013-07-11 15:22:45 -07:00
Karen Feng 15fd11d657 Removed redundant calls to request by logPage 2013-07-11 15:01:50 -07:00
Karen Feng 11872888ca Created getByteRange function for logs and log pages, removed lastNBytes function 2013-07-11 14:56:37 -07:00
Matei Zaharia 018d04c64e Merge pull request #684 from woggle/mesos-classloader 
Explicitly set class loader for MesosSchedulerDriver callbacks.
 2013-07-11 12:48:37 -07:00
Karen Feng e3a3fcf61b Scrollbar on log pages appear automatically 2013-07-11 12:16:38 -07:00
Karen Feng 044d4577ec Fixed capitalization of log page 2013-07-11 12:02:15 -07:00
Karen Feng 0ecc33f0c8 Added byte range, page title with log name, previous/next bytes buttons, initialization to end of log, large default buffer, buggy back to master link 2013-07-11 11:25:58 -07:00
Karen Feng 74bd3fc680 Added byte range on log pages 2013-07-10 15:44:28 -07:00
Karen Feng 24196c91f0 Changed buffer to 10,000 bytes, created scrollbar for fixed-height log 2013-07-10 15:27:52 -07:00
Karen Feng f5f3b272f8 Fixed mixup of start/end, moved more import files 2013-07-10 14:52:29 -07:00
Karen Feng 0d4580360b Fixed docstring of offsetBytes to match params and wrapped for 100+ character lines 2013-07-10 13:24:26 -07:00
Karen Feng 04263e4d46 Made some minor style changes 2013-07-10 13:15:42 -07:00
Karen Feng cfb6447ac4 Fixed for nonexistent bytes, added unit tests, changed stdout-page to stdout 2013-07-10 11:47:57 -07:00
seanm ee4ce2fc51 adding takeOrdered to java API 2013-07-10 10:46:04 -07:00
seanm 24705d0f46 adding takeOrdered() to RDD 2013-07-10 10:33:11 -07:00
Karen Feng 620a6974c6 Allows for larger files, refactors lastNBytes, removes old Log column, fixes imports, uses map 2013-07-10 10:20:53 -07:00
Karen Feng b6072b58bf Fixes style, makes "std__-page" consistent, reads only parts of files 2013-07-09 17:25:10 -07:00
Karen Feng 13fc6f248c Clean commit of log paging 2013-07-09 14:17:15 -07:00
Charles Reiss e47253e0cc Reset ClassLoader in MesosSchedulerBackend, too. (per review comments). 
Also set ClassLoader for all mesos callbacks, not just statusUpdate,
registered.
 2013-07-09 01:23:23 -07:00
Charles Reiss 8c1d1c98e0 Explicitly set class loader for MesosSchedulerDriver callbacks. 2013-07-08 12:25:46 -07:00
Shivaram Venkataraman 4af0d63cb1 Remove akka LogLevel fix as we no longer use spray 2013-07-07 10:42:43 -07:00
Shivaram Venkataraman a948f06725 Suppress log messages in sbt test with two changes: 
1. Set akka log level to ERROR before shutting down the actorSystem.
This avoids akka log messages (like Spray) from falling back to INFO
on the Stdout logger
2. Initialize netty to use SLF4J in LocalSparkContext. This ensures that
stack trace thrown during shutdown is handled by SLF4J instead of stdout
 2013-07-07 04:09:08 -07:00
Patrick Wendell 32b9d21a97 Fix occasional failure in UI listener. 
If a task fails before the metrics are initialized, it remains possible
that the metrics field will be `None`. This patch accounts for that possbility
by keeping metrics as an `Option` at all times.
 2013-07-06 16:40:02 -07:00
Matei Zaharia 1ffadb2d9e Merge remote-tracking branch 'pwendell/ui-updates' 
Conflicts:
	core/src/main/scala/spark/scheduler/DAGScheduler.scala
	core/src/main/scala/spark/util/AkkaUtils.scala
	pom.xml
 2013-07-06 15:51:41 -07:00