ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2891 commits 6 branches 181 tags 397 MiB
Commit graph

2891 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia 7902baddc7 Update ASM to version 4.0 2013-06-19 13:34:30 +02:00
Matei Zaharia dbfab49d2a Merge remote-tracking branch 'milliondreams/casdemo' 
Conflicts:
	project/SparkBuild.scala
 2013-06-18 14:55:31 +02:00
Matei Zaharia 73f4c7d2d1 Merge pull request #605 from esjewett/SPARK-699 
Add hBase example (retry of pull request #596)
 2013-06-18 04:21:17 -07:00
Matei Zaharia 9933836cc7 Merge pull request #647 from jerryshao/master 
Reduce ZippedPartitionsRDD's getPreferredLocations complexity from O(2^2n) to O(2^n)
 2013-06-18 02:41:10 -07:00
jerryshao 1e9269c3ee reduce ZippedPartitionsRDD's getPreferredLocations complexity 2013-06-18 09:49:06 +08:00
Matei Zaharia 2ab311f4ce Removed second version of junit test plugin from plugins.sbt 2013-06-18 00:40:25 +02:00
Matei Zaharia db42451a52 Merge pull request #643 from adatao/master 
Bug fix: Zero-length partitions result in NaN for overall mean & variance
 2013-06-17 15:26:36 -07:00
Matei Zaharia e82a2ffcc9 Merge pull request #653 from rxin/logging 
SPARK-781: Log the temp directory path when Spark says "Failed to create temp directory."
 2013-06-17 15:13:15 -07:00
Matei Zaharia ec193c7d89 Merge remote-tracking branch 'xiajunluan/xiajunluan' 
Conflicts:
	core/src/main/scala/spark/scheduler/cluster/TaskSetManager.scala
 2013-06-18 00:11:50 +02:00
Reynold Xin be3c406edf Fixed the typo pointed out by Matei. 2013-06-17 17:07:51 -04:00
Reynold Xin 1450296797 SPARK-781: Log the temp directory path when Spark says "Failed to create 
temp directory".
 2013-06-17 16:58:23 -04:00
Matei Zaharia e6d1277315 Merge pull request #638 from lyogavin/enhance_pipe 
Enhance pipe to support more features we can do in hadoop streaming
 2013-06-17 12:56:25 -07:00
Gavin Li 4508089fc3 refine comments and add sc.clean 2013-06-17 05:23:46 +00:00
Gavin Li 728665d34b Merge branch 'enhance_pipe' of ssh://github.com/lyogavin/spark into enhance_pipe 2013-06-17 00:23:29 +00:00
Yu L Li 33c55ec713 Merge remote branch 'upstream/master' into enhance_pipe 2013-06-17 00:21:26 +00:00
Gavin Li e6ae049283 Merge remote-tracking branch 'upstream1/master' into enhance_pipe 2013-06-16 22:53:39 +00:00
Gavin Li fb6d733fa8 update according to comments 2013-06-16 22:32:55 +00:00
Christopher Nguyen f91195cc15 Import just scala.math.abs rather than scala.math._ 2013-06-16 01:29:53 -07:00
Christopher Nguyen 5c886194e4 Move zero-length partition testing from JavaAPISuite.java to PartitioningSuite.scala 2013-06-16 01:23:48 -07:00
Christopher Nguyen 479442a9b9 Add zeroLengthPartitions() test to make sure, e.g., StatCounter.scala can handle empty partitions without incorrectly returning NaN 2013-06-15 17:35:55 -07:00
Matei Zaharia f961aac8b2 Merge pull request #649 from ryanlecompte/master 
Add top K method to RDD using a bounded priority queue
 2013-06-15 00:53:41 -07:00
ryanlecompte e8801d4490 use delegation for BoundedPriorityQueue, add Java API 2013-06-14 23:39:05 -07:00
Andrew xia 53add598f2 Update LocalSchedulerSuite to avoid using sleep for task launch 2013-06-15 01:46:13 +08:00
Matei Zaharia 6602d94973 Merge pull request #651 from rxin/groupbykey 
SPARK-772 / SPARK-774: groupByKey and cogroup should disable map side combine
 2013-06-14 10:41:31 -07:00
Reynold Xin 2cc188fd54 SPARK-774: cogroup should also disable map side combine by default 2013-06-14 00:10:54 -07:00
Reynold Xin 6738178d0d SPARK-772: groupByKey should disable map side combine. 2013-06-13 23:59:42 -07:00
ryanlecompte 93b3f5e535 drop unneeded ClassManifest implicit 2013-06-13 16:26:35 -07:00
ryanlecompte 44b8dbaede use Iterator.single(elem) instead of Iterator(elem) for improved performance based on scaladocs 2013-06-13 16:23:15 -07:00
Matei Zaharia d93851aedf Merge pull request #645 from pwendell/compression 
Adding compression to Hadoop save functions
 2013-06-13 13:38:45 -07:00
Rohit Rai b5b12823fa Fixing the style as per feedback 2013-06-13 14:05:46 +05:30
Matei Zaharia f1da591a8a Merge pull request #646 from markhamstra/jvmArgs 
Fixed jvmArgs in maven build.
 2013-06-12 17:55:08 -07:00
Mark Hamstra 3f96c6f27b Fixed jvmArgs in maven build. 2013-06-12 17:24:22 -07:00
ryanlecompte db5bca08ff add a new top K method to RDD using a bounded priority queue 2013-06-12 10:54:16 -07:00
Matei Zaharia dc4073654b Revert "Fix start-slave not passing instance number to spark-daemon." 
This reverts commit a674d67c0a.
 2013-06-11 00:08:02 -04:00
Matei Zaharia 0e94b734be Merge pull request #625 from stephenh/fix-start-slave 
Fix start-slave not passing instance number to spark-daemon.
 2013-06-10 13:00:31 -07:00
Patrick Wendell fd6148c8b2 Removing print statement 2013-06-10 10:27:25 -07:00
Andrew xia 190ec61799 change code style and debug info 2013-06-10 15:27:02 +08:00
Patrick Wendell ef14dc2e77 Adding Java-API version of compression codec 2013-06-09 18:09:46 -07:00
Patrick Wendell df592192e7 Monads FTW 2013-06-09 18:09:24 -07:00
Patrick Wendell 083a3485ab Clean extra whitespace 2013-06-09 11:49:33 -07:00
Patrick Wendell d1bbcebae5 Adding compression to Hadoop save functions 2013-06-09 11:39:35 -07:00
Matei Zaharia 74b91d53bc Merge pull request #629 from c0s/master 
Sometime Maven build runs out of PermGen space.
 2013-06-08 01:19:40 -07:00
Matei Zaharia b58a29295b Small formatting and style fixes 2013-06-07 22:51:28 -07:00
Matei Zaharia c8fc423bc2 Merge pull request #631 from jerryshao/master 
Fix block manager UI display issue when enable spark.cleaner.ttl
 2013-06-07 22:43:18 -07:00
Matei Zaharia c9ca0a4a58 Small code style fix to SchedulingAlgorithm.scala 2013-06-07 22:40:44 -07:00
Matei Zaharia 1ae60bcb36 Merge pull request #634 from xiajunluan/master 
[Spark-753] Fix ClusterSchedulSuite unit test failed
 2013-06-07 22:39:06 -07:00
Gavin Li e179ff8a32 update according to comments 2013-06-05 22:41:05 +00:00
Christopher Nguyen 9d35904357 In the current code, when both partitions happen to have zero-length, the return mean will be NaN. 
Consequently, the result of mean after reducing over all partitions will also be NaN,
which is not correct if there are partitions with non-zero length. This patch fixes this issue.
 2013-06-04 22:12:47 -07:00
Matei Zaharia fff3728552 Merge pull request #640 from pwendell/timeout-update 
Fixing bug in BlockManager timeout
 2013-06-04 16:09:50 -07:00
Patrick Wendell 061fd3ae36 Fixing bug in BlockManager timeout 2013-06-04 19:02:44 -04:00