ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
6602 commits 6 branches 181 tags 397 MiB
Commit graph

6602 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia b579b83277 Merge pull request #306 from kayousterhout/remove_pending 
Remove unused hasPendingTasks methods
 2013-12-27 22:09:04 -05:00
Kay Ousterhout b4619e509b Changed naming of StageCompleted event to be consistent 
The rest of the SparkListener events are named with "SparkListener"
as the prefix of the name; this commit renames the StageCompleted
event to SparkListenerStageCompleted for consistency.
 2013-12-27 17:45:20 -08:00
Kay Ousterhout e17d7518ab Removed unused OtherFailure TaskEndReason. 2013-12-27 15:51:27 -08:00
Kay Ousterhout 8419148e5f Remove unused hasPendingTasks methods 2013-12-27 15:19:42 -08:00
Patrick Wendell c8c8b42a6f Some notes and TODO about dependencies 2013-12-27 15:13:11 -08:00
Patrick Wendell 19672dca32 Merge pull request #305 from kayousterhout/line_spacing 
Fixed >100char lines in DAGScheduler.scala

There's no changed functionality here -- only line spacing and one grammatical fix in a comment.
 2013-12-27 13:37:10 -08:00
Tathagata Das 271e3237f3 Minor changes in comments and strings to address comments in PR 289. 2013-12-27 12:26:57 -08:00
Kay Ousterhout 0c71ffe924 Style fixes as per Reynold's review 2013-12-27 12:19:38 -08:00
Kay Ousterhout 8c81068e16 Fixed >100char lines in DAGScheduler.scala 2013-12-27 11:36:54 -08:00
Binh Nguyen 2c5bade4ee Fix failed unit tests 
Also clean up a bit.
 2013-12-27 11:24:30 -08:00
Kay Ousterhout baaabcedc9 Removed unused failed and causeOfFailure variables 2013-12-27 11:12:36 -08:00
Aaron Davidson 2a7b3511f4 Add Apache headers 2013-12-27 10:55:16 -08:00
Reynold Xin 7be1e57786 Merge pull request #298 from aarondav/minor 
Minor: Decrease margin of left side of Log page

Before
![before](https://f.cloud.github.com/assets/1400247/1812647/1a4be53e-6e87-11e3-9d5b-f851274be0e9.png)

After
![after](https://f.cloud.github.com/assets/1400247/1812648/1ca1ea2c-6e87-11e3-946c-31be9258f450.png)

It's a start anyway...
 2013-12-26 23:41:40 -10:00
Reynold Xin 7d811ba6f2 Merge pull request #302 from pwendell/SPARK-1007 
SPARK-1007: spark-class2.cmd should change SCALA_VERSION to be 2.10

Reported by Qiuzhuang Lian
 2013-12-26 23:39:58 -10:00
Reza Zadeh ae5102acc0 large scale considerations 2013-12-27 04:15:13 -05:00
Andrew Or d0cfbc41e2 Rename spark.shuffle.buffer variables 2013-12-27 00:07:09 -08:00
Andrew Or 8f3175773c Final cleanup 2013-12-26 23:40:08 -08:00
Aaron Davidson 1dc0440c1a Use real serializer & manual ordering 2013-12-26 23:40:08 -08:00
Aaron Davidson 0f66b7f2fc Return efficient iterator if no spillage happened 2013-12-26 23:40:08 -08:00
Andrew Or ec8c5dc644 Sort AppendOnlyMap in-place 2013-12-26 23:40:08 -08:00
Andrew Or a515706d9c Fix streaming JavaAPISuite again 2013-12-26 23:40:07 -08:00
Aaron Davidson 1ffe26c7c0 Fix streaming JavaAPISuite that depended on order 2013-12-26 23:40:07 -08:00
Aaron Davidson 0289eb752a Allow Product2 rather than just tuple kv pairs 2013-12-26 23:40:07 -08:00
Andrew Or 64b2d54a02 Move maps to util, and refactor more 2013-12-26 23:40:07 -08:00
Aaron Davidson 804beb43be SamplingSizeTracker + Map + test suite 2013-12-26 23:40:07 -08:00
Andrew Or 7ad4408255 New minor edits 2013-12-26 23:40:07 -08:00
Aaron Davidson fcc443b3db Minor cleanup for Scala style 2013-12-26 23:40:07 -08:00
Andrew Or 2a2ca2a661 Add toggle for ExternalAppendOnlyMap in Aggregator and CoGroupedRDD 2013-12-26 23:40:07 -08:00
Andrew Or 28685a4820 Provide for cases when mergeCombiners is not specified in ExternalAppendOnlyMap 2013-12-26 23:40:07 -08:00
Andrew Or 17def8cc11 Refactor ExternalAppendOnlyMap to take in KVC instead of just KV 2013-12-26 23:40:07 -08:00
Andrew Or 6a45ec1972 Working ExternalAppendOnlyMap for both CoGroupedRDDs and Aggregator 2013-12-26 23:40:07 -08:00
Andrew Or 97fbb3ec52 Working ExternalAppendOnlyMap for Aggregator, but not for CoGroupedRDD 2013-12-26 23:40:07 -08:00
Patrick Wendell 0cc1e0d43d SPARK-1007: spark-class2.cmd should change SCALA_VERSION to be 2.10 2013-12-26 23:21:08 -08:00
Reza Zadeh 642ab5c1e1 initial large scale testing begin 2013-12-27 01:51:19 -05:00
Reza Zadeh 3369c2d487 cleanup documentation 2013-12-27 00:41:46 -05:00
Reza Zadeh bdb5037987 add all tests 2013-12-27 00:36:41 -05:00
Reza Zadeh fa1e8d8cbf test for truncated svd 2013-12-27 00:34:59 -05:00
Reza Zadeh 16de5268e3 full rank matrix test added 2013-12-26 23:21:57 -05:00
Tathagata Das 6e43039614 Refactored streaming project to separate out the twitter functionality. 2013-12-26 18:02:49 -08:00
Lian, Cheng d7086dc28a Added Apache license header to NaiveBayesSuite 2013-12-27 08:20:41 +08:00
Matei Zaharia 5e69fc5bb4 Merge pull request #295 from markhamstra/JobProgressListenerNPE 
Avoid a lump of coal (NPE) in JobProgressListener's stocking.
 2013-12-26 19:10:39 -05:00
Patrick Wendell 55c8bb741c Intermediate clean-up of tests to appease jenkins 2013-12-26 15:43:15 -08:00
Aaron Davidson 4f2fb761b0 Decrease margin of left side of log page 2013-12-26 15:38:45 -08:00
Reza Zadeh fe1a132d40 Main method added for svd 2013-12-26 18:13:21 -05:00
Reza Zadeh 1a21ba2967 new main file 2013-12-26 18:09:33 -05:00
Patrick Wendell 5c1b4f6405 Minor fixes 2013-12-26 14:39:39 -08:00
Reza Zadeh 6c3674cd23 Object to hold the svd methods 2013-12-26 17:39:25 -05:00
Tathagata Das 5fde4566ea Added Apache boilerplate and class docs to PartitionerAwareUnionRDD. 2013-12-26 14:33:37 -08:00
Tathagata Das 577c8cc834 Removed unncessary options from WindowedDStream. 2013-12-26 14:17:16 -08:00
Reza Zadeh 6e740cc901 Some documentation 2013-12-26 16:12:40 -05:00