ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3037 commits 6 branches 181 tags 397 MiB
Commit graph

3037 commits

This branch
This branch
All branches
Author SHA1 Message Date
Andy Konwinski 446b801b3b Fixing typos pointed out by Matei 2013-03-20 17:30:31 -07:00
Stephen Haberman 4f4215311a Merge branch 'master' into volatile 2013-03-20 15:37:10 -05:00
Matei Zaharia ca4d083ec8 Merge pull request #528 from MLnick/java-examples 
[SPARK-707] Adding Java versions of Pi, LogQuery and K-Means examples
 2013-03-20 11:22:36 -07:00
Matei Zaharia b812e6b7bb Merge pull request #526 from markhamstra/foldByKey 
Add foldByKey
 2013-03-20 11:21:02 -07:00
Nick Pentreath 5a378a1945 Merge branch 'java-examples' of https://github.com/MLnick/spark into java-examples 2013-03-20 09:57:00 +02:00
Nick Pentreath 52398cc1a3 Java indentation 4 --> 2 spaces 2013-03-20 09:55:42 +02:00
Nick Pentreath 7a443bb7d2 Java indentation 2 --> 4 spaces 2013-03-20 09:49:57 +02:00
Reynold Xin d48ee7e55e Merge branch 'master' of github.com:mesos/spark into cogroup 2013-03-20 14:00:28 +08:00
Reynold Xin 00a11304fd Added mapSideCombine flag to CoGroupedRDD. Added unit test for 
CoGroupedRDD.
 2013-03-20 13:49:51 +08:00
Matei Zaharia 945d1e720e Merge pull request #536 from sasurfer/master 
CoalescedRDD for many partitions
 2013-03-19 21:59:06 -07:00
Matei Zaharia 1cbbe94ac1 Merge pull request #534 from stephenh/removetrycatch 
Remove try/catch block that can't be hit.
 2013-03-19 21:34:34 -07:00
Matei Zaharia 71e53f8894 Merge pull request #537 from wishbear/configurableInputFormat 
call setConf from input format if it is Configurable
 2013-03-19 21:31:41 -07:00
Nick Pentreath 9fa47a2039 A few cosmetic changes for JavaKMeans 2013-03-19 15:31:03 +02:00
Nick Pentreath 568ddf7330 Adding Java K-Means example 2013-03-19 15:29:22 +02:00
Andrey Kouznetsov bd167f83b0 call setConf from input format if it is Configurable 2013-03-19 17:15:15 +04:00
Giovanni Delussu aceae029f7 CoalescedRDD changed to work with a big number of partitions both in the original and the new coalesced RDD. 
The limitation was in the range that Scala.Int can represent.
 2013-03-19 11:25:45 +01:00
Reynold Xin ba9d00c44a Merge branch 'master' into graph 
Conflicts:
	run2.cmd
 2013-03-18 18:30:14 +08:00
Stephen Haberman fb34967815 Remove try/catch block that can't be hit. 2013-03-18 01:55:50 -05:00
Andy Konwinski ad7f0452ab Adds page to docs about building using Maven. 
Adds links to new instructions in:
* The main Spark project README.md
* The docs nav menu called "More"
* The docs Overview page under the "Building" and "Where to Go from Here" sections
 2013-03-17 15:02:40 -07:00
Nick Pentreath b990caeb80 Changes to more closely match line length limit style 2013-03-17 20:03:27 +02:00
Mark Hamstra ab33e27cc9 constructorOfA -> constructA in doc comments 2013-03-16 15:29:15 -07:00
Mark Hamstra 9784fc1fcd fix wayward comma in doc comment 2013-03-16 15:25:02 -07:00
Mark Hamstra 32979b5e7d whitespace 2013-03-16 13:36:46 -07:00
Mark Hamstra ca9f81e8fc refactor foldByKey to use combineByKey 2013-03-16 13:31:01 -07:00
Mark Hamstra 1fb192ef40 Merge branch 'master' of https://github.com/mesos/spark into foldByKey 2013-03-16 12:17:13 -07:00
Mark Hamstra 80fc8c82ed _With[Matei] 2013-03-16 12:16:29 -07:00
Mark Hamstra 38454c4aed Merge branch 'master' of https://github.com/mesos/spark into WithThing 2013-03-16 11:54:44 -07:00
Matei Zaharia c1e9cdc49f Merge pull request #525 from stephenh/subtractByKey 
Add PairRDDFunctions.subtractByKey.
 2013-03-16 11:47:45 -07:00
seanm d61978d0ab keeping JavaStreamingContext in sync with StreamingContext + adding comments for better clarity 2013-03-15 23:36:52 -06:00
Mark Hamstra ef75be3bf7 Merge branch 'master' of https://github.com/mesos/spark into foldByKey 2013-03-15 21:41:24 -07:00
Andrew xia 5892393140 refactor fair scheduler implementation 
1.Chage "pool" properties to be the memeber of ActiveJob
2.Abstract the Schedulable of Pool and TaskSetManager
3.Abstract the FIFO and FS comparator algorithm
4.Miscellaneous changing of class define and construction
 2013-03-16 11:13:38 +08:00
Matei Zaharia cdbfd1e196 Merge pull request #516 from squito/fix_local_metrics 
Fix local metrics
 2013-03-15 15:13:28 -07:00
Matei Zaharia f9fa2add5c Merge pull request #530 from mbautin/master-update-log4j-and-make-compile-in-IntelliJ 
Add a log4j compile dependency to fix build in IntelliJ
 2013-03-15 15:12:43 -07:00
Mikhail Bautin 7fd2708eda Add a log4j compile dependency to fix build in IntelliJ 
Also rename parent project to spark-parent (otherwise it shows up as
"parent" in IntelliJ, which is very confusing).
 2013-03-15 11:41:51 -07:00
Mark Hamstra 1a4070477d whitespace cleanup 2013-03-15 11:28:28 -07:00
Mark Hamstra 857010392b Fuller implementation of foldByKey 2013-03-15 10:56:05 -07:00
Nick Pentreath 13757b1198 Adding Java versions of Pi and LogQuery 2013-03-15 10:52:01 +02:00
seanm 33fa1e7e4a removing dependency on ZookeeperConsumerConnector + purging last relic of kafka reliability that never solidified (ie- setOffsets) 2013-03-15 00:10:13 -06:00
seanm d069283211 fixing memory leak in kafka MessageHandler 2013-03-14 23:45:33 -06:00
seanm cfa8e769a8 KafkaInputDStream improvements. Allows more Kafka configurability 2013-03-14 23:45:19 -06:00
Mark Hamstra 16a4ca4537 restrict V type of foldByKey in order to retain ClassManifest; added foldByKey to Java API and test 2013-03-14 13:58:37 -07:00
Mark Hamstra b1422cbdd5 added foldByKey 2013-03-14 12:59:58 -07:00
Stephen Haberman 7786881f47 Fix tabs that snuck in. 2013-03-14 14:57:12 -05:00
Stephen Haberman 7d8bb4df3a Allow subtractByKey's other argument to have a different value type. 2013-03-14 14:44:15 -05:00
Stephen Haberman 4632c45af1 Finished subtractByKeys. 2013-03-14 10:35:34 -05:00
Matei Zaharia 4032beba49 Merge pull request #521 from stephenh/earlyclose 
Close the reader in HadoopRDD as soon as iteration end.
 2013-03-13 19:29:46 -07:00
Matei Zaharia 3c97276a4a Merge pull request #524 from andyk/master 
Fix broken link to YARN documentation
 2013-03-13 19:25:08 -07:00
Matei Zaharia 1c3d98197b Merge pull request #517 from Reinvigorate/sm-build-fixes 
Build fixes for streaming /w SBT
 2013-03-13 19:23:48 -07:00
Stephen Haberman 63fe225587 Simplify SubtractedRDD in preparation from subtractByKey. 2013-03-13 17:17:34 -05:00
Andy Konwinski c9097628fc Fix broken link to YARN documentation. 2013-03-13 14:51:13 -07:00