ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2773 commits 6 branches 181 tags 397 MiB
Commit graph

2773 commits

This branch
This branch
All branches
Author SHA1 Message Date
Reynold Xin 9e64396ca4 Cleaned up the Java files from Shane's PR. 2013-05-07 18:30:54 -07:00
Reynold Xin 0e5cc30868 Cleaned up BlockManager and BlockFetcherIterator from Shane's PR. 2013-05-07 18:18:24 -07:00
Reynold Xin 8b79485171 Moved BlockFetcherIterator to its own file. 2013-05-07 17:02:32 -07:00
Matei Zaharia a8e13f6a0b Merge branch 'master' of github.com:mesos/spark 2013-05-07 19:06:57 -04:00
Reynold Xin 90577ada69 Merge branch 'shuffle-performance-fix-0.7' of github.com:shane-huang/spark into shufflemerge 
Conflicts:
	core/src/main/scala/spark/storage/BlockManager.scala
	core/src/main/scala/spark/storage/DiskStore.scala
	project/SparkBuild.scala
 2013-05-07 15:56:19 -07:00
Jey Kottalam aacca1b8a8 Update Maven build to Scala 2.9.3 2013-05-07 14:39:44 -07:00
Ethan Jewett a3d5f92210 Switch to using SparkContext method to create RDD 2013-05-07 11:43:06 -05:00
Jey Kottalam 4d8919d330 Update Maven build to Scala 2.9.3 2013-05-06 17:05:50 -07:00
Reynold Xin 64d4d2b036 Added tests for joins, cogroups, and unions for EmptyRDD. 2013-05-06 16:30:46 -07:00
Reynold Xin 0fd84965f6 Added EmptyRDD. 2013-05-06 15:40:34 -07:00
Reynold Xin e93805f2b1 Merge branch 'master' of github.com:mesos/spark 2013-05-06 15:39:38 -07:00
Imran Rashid 22a5063ae4 switch from separating appUI host & port to combining into just appUiUrl 2013-05-05 12:19:11 -07:00
Josh Rosen e014c1d1cb Fix SPARK-670: EC2 start command should require -i option. 2013-05-05 11:30:36 -07:00
Matei Zaharia 7af92f248b Merge pull request #597 from JoshRosen/webui-fixes 
Two minor bug fixes for Spark Web UI
 2013-05-04 22:29:17 -07:00
Matei Zaharia c74ce606a3 Merge pull request #598 from rxin/blockmanager 
Fixed flaky unpersist test in DistributedSuite.
 2013-05-04 22:26:35 -07:00
Reynold Xin 0a2bed356b Fixed flaky unpersist test in DistributedSuite. 2013-05-04 21:50:08 -07:00
Reynold Xin 62a077cd08 Merge branch 'unpersist-test' of github.com:shivaram/spark into blockmanager 2013-05-04 21:49:50 -07:00
Josh Rosen 42b1953c53 Fix SPARK-630: app details page shows finished executors as running. 2013-05-04 18:34:47 -07:00
Josh Rosen c0688451a6 Fix wrong closing tags in web UI HTML. 2013-05-04 18:34:46 -07:00
Josh Rosen d48e9fde01 Fix SPARK-629: weird number of cores in job details page. 2013-05-04 18:34:45 -07:00
Ethan Jewett 7cff7e7897 Fix indents and mention other configuration options 2013-05-04 14:56:55 -05:00
Ethan Jewett 9290f16430 Remove unnecessary column family config 2013-05-04 12:39:14 -05:00
Ethan Jewett 02e8cfa617 HBase example 2013-05-04 12:31:30 -05:00
Mridul Muralidharan 25198d7e9e Merge branch 'master' of github.com:mridulm/spark 2013-05-04 20:45:56 +05:30
Mridul Muralidharan 5b011d18d7 Merge from master 2013-05-04 20:41:27 +05:30
Mridul Muralidharan edb57c8331 Add support for instance local in getPreferredLocations of ZippedPartitionsBaseRDD. Add comments to both ZippedPartitionsBaseRDD and ZippedRDD to better describe the potential problem with the approach 2013-05-04 19:47:45 +05:30
Matei Zaharia 3bf2c868c3 Merge pull request #594 from shivaram/master 
Add zip partitions to Java API
 2013-05-03 18:27:30 -07:00
Matei Zaharia 2484ad70dc Merge pull request #587 from rxin/blockmanager 
A set of shuffle map output related changes
 2013-05-03 17:08:55 -07:00
Shivaram Venkataraman 2274ad0786 Fix flaky test by changing catch and adding sleep 2013-05-03 16:35:35 -07:00
Shivaram Venkataraman bb8a434f9d Add zipPartitions to Java API. 2013-05-03 15:14:02 -07:00
Imran Rashid 6fae936088 applications (aka drivers) send their webUI address to master when registering so it can be displayed in the master web ui 2013-05-03 12:59:10 -07:00
Mridul Muralidharan ea2a6f91d3 pull from master 2013-05-04 00:35:59 +05:30
Reynold Xin 93091f6936 Merge branch 'master' of github.com:mesos/spark into blockmanager 2013-05-03 01:02:32 -07:00
Reynold Xin 2bc895a829 Updated according to Matei's code review comment. 2013-05-03 01:02:16 -07:00
Mridul Muralidharan 11589c39d9 Fix ZippedRDD as part Matei's suggestion 2013-05-03 12:23:30 +05:30
Matei Zaharia 6fe9d4e61e Merge pull request #592 from woggling/localdir-fix 
Don't accept generated local directory names that can't be created
 2013-05-02 21:33:56 -07:00
Matei Zaharia 538ee755b4 Merge pull request #581 from jerryshao/master 
fix [SPARK-740] block manage UI throws exception when enabling Spark Streaming
 2013-05-02 09:01:42 -07:00
Charles Reiss c847dd3da2 Don't accept generated temp directory names that can't be created successfully. 2013-05-01 23:19:10 -07:00
Reynold Xin d13784b81e Merge branch 'master' of github.com:mesos/spark 2013-05-01 22:50:25 -07:00
Matei Zaharia 9abcbcc76d Merge pull request #591 from rxin/removerdd 
RDD.unpersist: probably the most desired feature of Spark
 2013-05-01 22:45:10 -07:00
Reynold Xin 4a31877408 Added the unpersist api to JavaRDD. 2013-05-01 20:31:54 -07:00
Reynold Xin 98df9d2853 Added removeRdd function in BlockManager. 2013-05-01 20:17:09 -07:00
Mridul Muralidharan dfde9ce9dd comment out debug versions of checkHost, etc from Utils - which were used to test 2013-05-02 07:41:33 +05:30
Mridul Muralidharan 1b5aaeadc7 Integrate review comments 2 2013-05-02 07:30:06 +05:30
jerryshao c047f0e3ad filter out Spark streaming block RDD and sort RDDInfo with id 2013-05-02 09:48:32 +08:00
Mridul Muralidharan 609a817f52 Integrate review comments on pull request 2013-05-02 06:44:33 +05:30
Jey Kottalam 207afe4088 Remove spark-repl's extraneous dependency on spark-streaming 2013-05-01 16:57:31 -07:00
Reynold Xin 204eb32e14 Changed the type of the persistentRdds hashmap back to 
TimeStampedHashMap.
 2013-05-01 16:14:58 -07:00
Reynold Xin 34637b97ec Added SparkContext.cleanup back. Not sure why it was removed before ... 2013-05-01 16:12:37 -07:00
Reynold Xin 3227ec8edd Cleaned up Ram's code. Moved SparkContext.remove to RDD.unpersist. 
Also updated unit tests to make sure they are properly testing for
concurrency.
 2013-05-01 16:07:44 -07:00