ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2742 commits 6 branches 181 tags 397 MiB
Commit graph

1364 commits

Author SHA1 Message Date
Reynold Xin 203d7b7c14 Merge pull request #593 from squito/driver_ui_link 
Master UI has link to Application UI
 2013-05-15 00:47:20 -07:00
Reynold Xin f9d40a5848 Added a comment in JdbcRDD for example usage. 2013-05-14 23:29:57 -07:00
Reynold Xin 404f9ff617 Added derby dependency to Maven pom files for the JDBC Java test. 2013-05-14 23:28:34 -07:00
Reynold Xin 81ad2fa331 Merge branch 'jdbc' of github.com:koeninger/spark 
Conflicts:
	project/SparkBuild.scala
 2013-05-14 23:12:00 -07:00
Cody Koeninger b16c4896f6 add test for JdbcRDD using embedded derby, per rxin suggestion 2013-05-14 23:44:04 -05:00
Matei Zaharia 016ac86830 Merge pull request #601 from rxin/emptyrdd-master 
EmptyRDD (master branch 0.8)
 2013-05-13 21:45:36 -07:00
Matei Zaharia 4b354e0a08 Merge pull request #589 from mridulm/master 
Add support for instance local scheduling
 2013-05-13 17:39:19 -07:00
Patrick Wendell 7f0833647b Capturing class name 2013-05-12 07:54:03 -07:00
Patrick Wendell 72b9c4cb6e Small fix 2013-05-11 23:53:50 -07:00
Patrick Wendell 1c15b85051 Removing import 2013-05-11 23:52:53 -07:00
Patrick Wendell 059ab88754 Changing technique to use same code path in all cases 2013-05-11 23:50:54 -07:00
Cody Koeninger 3da2305ed0 code cleanup per rxin comments 2013-05-11 23:59:07 -05:00
Patrick Wendell a5c28bb888 Removing unnecessary map 2013-05-11 14:20:39 -07:00
Patrick Wendell 0345954530 SPARK-738: Spark should detect and squash nonserializable exceptions 2013-05-11 14:17:09 -07:00
Mark Hamstra 6e6b3e0d7e Actually use the cleaned closure in foreachPartition 2013-05-10 13:02:34 -07:00
Mridul Muralidharan b05c9d22d7 Remove explicit hardcoding of yarn-standalone as args(0) if it is missing. 2013-05-09 18:49:12 +05:30
Imran Rashid 0ab818d508 fix linebreak 2013-05-09 00:38:59 -07:00
Jey Kottalam aacca1b8a8 Update Maven build to Scala 2.9.3 2013-05-07 14:39:44 -07:00
Reynold Xin 64d4d2b036 Added tests for joins, cogroups, and unions for EmptyRDD. 2013-05-06 16:30:46 -07:00
Reynold Xin 0fd84965f6 Added EmptyRDD. 2013-05-06 15:40:34 -07:00
Imran Rashid 22a5063ae4 switch from separating appUI host & port to combining into just appUiUrl 2013-05-05 12:19:11 -07:00
Matei Zaharia 7af92f248b Merge pull request #597 from JoshRosen/webui-fixes 
Two minor bug fixes for Spark Web UI
 2013-05-04 22:29:17 -07:00
Reynold Xin 0a2bed356b Fixed flaky unpersist test in DistributedSuite. 2013-05-04 21:50:08 -07:00
Reynold Xin 62a077cd08 Merge branch 'unpersist-test' of github.com:shivaram/spark into blockmanager 2013-05-04 21:49:50 -07:00
Josh Rosen 42b1953c53 Fix SPARK-630: app details page shows finished executors as running. 2013-05-04 18:34:47 -07:00
Josh Rosen c0688451a6 Fix wrong closing tags in web UI HTML. 2013-05-04 18:34:46 -07:00
Josh Rosen d48e9fde01 Fix SPARK-629: weird number of cores in job details page. 2013-05-04 18:34:45 -07:00
Mridul Muralidharan 25198d7e9e Merge branch 'master' of github.com:mridulm/spark 2013-05-04 20:45:56 +05:30
Mridul Muralidharan 5b011d18d7 Merge from master 2013-05-04 20:41:27 +05:30
Mridul Muralidharan edb57c8331 Add support for instance local in getPreferredLocations of ZippedPartitionsBaseRDD. Add comments to both ZippedPartitionsBaseRDD and ZippedRDD to better describe the potential problem with the approach 2013-05-04 19:47:45 +05:30
Matei Zaharia 3bf2c868c3 Merge pull request #594 from shivaram/master 
Add zip partitions to Java API
 2013-05-03 18:27:30 -07:00
Shivaram Venkataraman 2274ad0786 Fix flaky test by changing catch and adding sleep 2013-05-03 16:35:35 -07:00
Shivaram Venkataraman bb8a434f9d Add zipPartitions to Java API. 2013-05-03 15:14:02 -07:00
Imran Rashid 6fae936088 applications (aka drivers) send their webUI address to master when registering so it can be displayed in the master web ui 2013-05-03 12:59:10 -07:00
Mridul Muralidharan ea2a6f91d3 pull from master 2013-05-04 00:35:59 +05:30
Reynold Xin 93091f6936 Merge branch 'master' of github.com:mesos/spark into blockmanager 2013-05-03 01:02:32 -07:00
Reynold Xin 2bc895a829 Updated according to Matei's code review comment. 2013-05-03 01:02:16 -07:00
Mridul Muralidharan 11589c39d9 Fix ZippedRDD as part Matei's suggestion 2013-05-03 12:23:30 +05:30
Matei Zaharia 6fe9d4e61e Merge pull request #592 from woggling/localdir-fix 
Don't accept generated local directory names that can't be created
 2013-05-02 21:33:56 -07:00
Matei Zaharia 538ee755b4 Merge pull request #581 from jerryshao/master 
fix [SPARK-740] block manage UI throws exception when enabling Spark Streaming
 2013-05-02 09:01:42 -07:00
Charles Reiss c847dd3da2 Don't accept generated temp directory names that can't be created successfully. 2013-05-01 23:19:10 -07:00
Reynold Xin 4a31877408 Added the unpersist api to JavaRDD. 2013-05-01 20:31:54 -07:00
Reynold Xin 98df9d2853 Added removeRdd function in BlockManager. 2013-05-01 20:17:09 -07:00
Mridul Muralidharan dfde9ce9dd comment out debug versions of checkHost, etc from Utils - which were used to test 2013-05-02 07:41:33 +05:30
Mridul Muralidharan 1b5aaeadc7 Integrate review comments 2 2013-05-02 07:30:06 +05:30
jerryshao c047f0e3ad filter out Spark streaming block RDD and sort RDDInfo with id 2013-05-02 09:48:32 +08:00
Mridul Muralidharan 609a817f52 Integrate review comments on pull request 2013-05-02 06:44:33 +05:30
Reynold Xin 204eb32e14 Changed the type of the persistentRdds hashmap back to 
TimeStampedHashMap.
 2013-05-01 16:14:58 -07:00
Reynold Xin 34637b97ec Added SparkContext.cleanup back. Not sure why it was removed before ... 2013-05-01 16:12:37 -07:00
Reynold Xin 3227ec8edd Cleaned up Ram's code. Moved SparkContext.remove to RDD.unpersist. 
Also updated unit tests to make sure they are properly testing for
concurrency.
 2013-05-01 16:07:44 -07:00