ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3283 commits 6 branches 181 tags 397 MiB
Commit graph

1654 commits

Author SHA1 Message Date
Dmitriy Lyubimov 984b56155a changing approaches for parallelize(): java serialization needs to avoid writing headers! 2013-07-30 17:36:59 -07:00
Dmitriy Lyubimov ef9529a943 refactoring using writeByteBuffer() from Utils. 2013-07-30 16:24:23 -07:00
Dmitriy Lyubimov 43394b9a6d fixing formatting 2013-07-30 16:13:41 -07:00
Dmitriy Lyubimov 13a9d66645 adding === 2013-07-30 16:10:55 -07:00
Dmitriy Lyubimov 1bca91633e + bug fixes; 
test added

Conflicts:

	core/src/test/scala/spark/KryoSerializerSuite.scala
 2013-07-30 11:04:11 -07:00
Dmitriy Lyubimov 8e5cd041bb initial externalization of ParallelCollectionRDD's split 2013-07-29 19:02:53 -07:00
Dmitriy Lyubimov 23f3e0f117 mixing in SharedSparkContext for the kryo-collect test 2013-07-26 19:15:11 -07:00
Dmitriy Lyubimov 72bac09c42 Leaking spark context in the test 2013-07-23 15:19:07 -07:00
Dmitriy Lyubimov ef82ff8564 Merge branch 'master' into SPARK-826 
Conflicts:
	core/src/main/scala/spark/scheduler/local/LocalScheduler.scala
 2013-07-23 13:43:00 -07:00
Dmitriy Lyubimov 310e73d566 style 2013-07-23 13:23:25 -07:00
Matei Zaharia f369e0e51b Merge pull request #720 from ooyala/2013-07/persistent-rdds-api 
Add a public method getCachedRdds to SparkContext
 2013-07-23 13:22:27 -07:00
Dmitriy Lyubimov ac60d06381 Re-working in terms of changes to TaskSetManager. Verified with Standalone and Local mode. 2013-07-23 13:13:19 -07:00
Evan Chan efd6418c1b Move getPersistentRDDs testing to a new Suite 2013-07-23 10:40:41 -07:00
Evan Chan 4830e22562 Rename method per rxin feedback 2013-07-23 09:50:13 -07:00
Evan Chan 2c2bfbe294 Add toMap method to TimeStampedHashMap and use it 2013-07-23 01:36:44 -07:00
Matei Zaharia 401aac8b18 Merge pull request #719 from karenfeng/ui-808 
Creates Executors tab for Jobs UI
 2013-07-22 16:57:16 -07:00
Karen Feng 872c97ad82 Split task columns, memory columns sort by numeric value 2013-07-22 16:54:37 -07:00
Matei Zaharia ea1cfabfdd Merge branch 'master' of github.com:mesos/spark 2013-07-22 16:22:02 -07:00
Matei Zaharia 8e38e77232 Fix a test that was using an outdated config setting 2013-07-22 16:05:32 -07:00
Karen Feng 2eea974795 Executors UI now calls executor ID from TaskInfo instead of TaskMetrics 2013-07-22 15:15:54 -07:00
Dmitriy Lyubimov 8ca0c31944 removing non-pertinent comment 2013-07-22 14:48:46 -07:00
Dmitriy Lyubimov b4b230e606 Fixing for LocalScheduler with test, that much works .. 2013-07-22 14:42:47 -07:00
Karen Feng 85c4d7bf3b Shows number of complete/total/failed tasks (bug: failed tasks assigned to null executor) 2013-07-22 14:35:47 -07:00
Josh Rosen f649dabb4a Fix bug: DoubleRDDFunctions.sampleStdev() computed non-sample stdev(). 
Update JavaDoubleRDD to add new methods and docs.

Fixes SPARK-825.
 2013-07-22 13:21:48 -07:00
Karen Feng 8901f379c9 Fixed memory used/remaining/total bug 2013-07-22 09:58:03 -07:00
Karen Feng 636b19f833 Merge branch 'master' of https://github.com/mesos/spark into ui-808 2013-07-22 09:53:26 -07:00
Evan Chan 0337d88321 Add a public method getCachedRdds to SparkContext 2013-07-21 18:26:14 -07:00
Karen Feng 865dc63bac Changed table format for executors 2013-07-19 15:57:01 -07:00
Karen Feng 81bb5dc640 Creates Executors tab for application with RDD block and memory/disk used, solves SPARK-808 2013-07-19 14:08:30 -07:00
Konstantin Boudnik cfce9a6a36 Regression: default webui-port can't be set via command line "--webui-port" anymore 2013-07-19 14:00:58 -07:00
Liang-Chi Hsieh 4530e8a9bf fix typo. 2013-07-20 00:04:25 +08:00
Liang-Chi Hsieh aa6f83289b A better fix for giving local jars unde Yarn mode. 2013-07-19 22:25:28 +08:00
Liang-Chi Hsieh a613628c50 Do not copy local jars given to SparkContext in yarn mode since the Context is not running on local. This bug causes failure when jars can not be found. Example codes (such as spark.examples.SparkPi) can not work without this fix under yarn mode. 2013-07-19 16:59:12 +08:00
Matei Zaharia af3c9d5042 Add Apache license headers and LICENSE and NOTICE files 2013-07-16 17:21:33 -07:00
Matei Zaharia b1f9f64743 Merge branch 'master' of github.com:mesos/spark 2013-07-16 11:01:53 -07:00
Matei Zaharia 5c388808a8 SPARK-814: Result stages should be named after action 2013-07-16 11:01:14 -07:00
Matei Zaharia f347cc3f65 Fix deprecation warning and style issues 2013-07-16 10:53:30 -07:00
Reynold Xin 69316603d6 Throw a more meaningful message when runJob is called to launch tasks on non-existent partitions. 2013-07-15 22:50:11 -07:00
Karen Feng 6dc7c9bfb1 Removed job UI column, linked description to job UI 2013-07-15 16:33:50 -07:00
Karen Feng fbf5aa761e Removed log message, added field in master UI to link to log UI 2013-07-15 15:50:03 -07:00
Karen Feng eac381a957 Merge branch 'ui-802' of https://github.com/karenfeng/spark into ui-802 2013-07-15 15:48:44 -07:00
Karen Feng 3955711250 Added field to master UI with link to job UI 2013-07-15 15:47:21 -07:00
Karen Feng 0d78b6d9cd Links to job UI from standalone deploy cluster web UI: fixes SPARK-802 2013-07-15 13:47:38 -07:00
Karen Feng b2aaa1199e Adds app name in HTML page titles on job web UI: fixes SPARK-806 2013-07-15 11:44:42 -07:00
Matei Zaharia d47c16f78d Add an option to disable reference tracking in Kryo 2013-07-15 01:55:54 +00:00
Matei Zaharia c7877d5e16 Merge pull request #689 from BlackNiuza/application_status 
Bug fix: SPARK-796
 2013-07-14 12:58:13 -07:00
Matei Zaharia 10c05937bd Merge pull request #699 from pwendell/ui-env 
Add `Environment` tab to SparkUI.
 2013-07-14 11:45:18 -07:00
Patrick Wendell 4883586838 Responding to Matei's review 2013-07-14 10:37:26 -07:00
BlackNiuza 00556a94c9 add spaces before curly braces and after for if conditions 2013-07-14 17:04:53 +08:00
Matei Zaharia b91a218cea Cosmetic fixes to web UI 2013-07-14 07:31:33 +00:00