ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
3828 commits 6 branches 181 tags 397 MiB
Commit graph

3828 commits

This branch
This branch
All branches
Author SHA1 Message Date
Patrick Wendell 4879685910 Merge remote-tracking branch 'mesos/master' into ec2-updates 2013-08-24 14:50:58 -07:00
Matei Zaharia d282c1ebbb Merge pull request #860 from jey/sbt-ide-fixes 
Fix IDE project generation under SBT
 2013-08-23 11:20:20 -07:00
Jey Kottalam a9db1b7b6e Upgrade SBT IDE project generators 2013-08-23 10:27:18 -07:00
Jey Kottalam b7f9e6374a Fix SBT generation of IDE project files 2013-08-23 10:26:37 -07:00
Matei Zaharia 5a6ac12840 Merge pull request #701 from ScrapCodes/documentation-suggestions 
Documentation suggestions for spark streaming.
 2013-08-22 22:08:03 -07:00
Prashant Sharma 2bc348e92c Linking custom receiver guide 2013-08-23 09:44:02 +05:30
Prashant Sharma 3049415e24 Corrections in documentation comment 2013-08-23 09:40:28 +05:30
Prashant Sharma 39a1d58da4 Improved documentation for spark custom receiver 2013-08-23 09:38:50 +05:30
Matei Zaharia 215c13dd41 Fix code style and a nondeterministic RDD issue in ALS 2013-08-22 16:13:46 -07:00
Matei Zaharia 46ea0c1b47 Merge pull request #814 from holdenk/master 
Create less instances of the random class during ALS initialization.
 2013-08-22 15:57:28 -07:00
Matei Zaharia 9ac3d62cac Merge pull request #856 from jey/sbt-fix-hadoop-0.23.9 
Re-add removed dependency to fix build under Hadoop 0.23.9
 2013-08-22 15:51:10 -07:00
Jey Kottalam 281b6c5f28 Re-add removed dependency on 'commons-daemon' 
Fixes SBT build under Hadoop 0.23.9 and 2.0.4
 2013-08-22 15:45:45 -07:00
Matei Zaharia ae8ba83ef2 Merge pull request #855 from jey/update-build-docs 
Update build docs
 2013-08-22 10:14:54 -07:00
Matei Zaharia 8a36fd09dd Merge pull request #854 from markhamstra/pomUpdate 
Synced sbt and maven builds to use the same dependencies, etc.
 2013-08-22 10:13:35 -07:00
Matei Zaharia c2d00f12e2 Merge pull request #832 from alig/coalesce 
Coalesced RDD with locality
 2013-08-22 10:13:03 -07:00
Jey Kottalam 9a90667d09 Increase ReservedCodeCacheSize to 256m 2013-08-21 21:15:28 -07:00
Jey Kottalam 0087b43e9c Use Hadoop 1.2.1 in application example 2013-08-21 21:15:00 -07:00
Jey Kottalam 54e9379de2 Revert "Allow build configuration to be set in conf/spark-env.sh" 
This reverts commit 66e7a38a32.
 2013-08-21 21:13:34 -07:00
Matei Zaharia e6d66c8abd Merge pull request #853 from AndreSchumacher/double_rdd 
Implementing SPARK-838: Add DoubleRDDFunctions methods to PySpark
 2013-08-21 17:44:31 -07:00
Jey Kottalam f9cc1fbf27 Remove references to unsupported Hadoop versions 2013-08-21 17:14:36 -07:00
Andre Schumacher 76077bf9f4 Implementing SPARK-838: Add DoubleRDDFunctions methods to PySpark 2013-08-21 17:05:58 -07:00
Patrick Wendell c02585ea13 Make initial connection failure message less daunting. 
Right now it seems like something has gone wrong when this message is printed out.
Instead, this is a normal condition. So I changed the message a bit.
 2013-08-21 15:45:45 -07:00
Patrick Wendell 6be6b71c8c Merge branch 'master' into ec2-updates 
Conflicts:
	ec2/spark_ec2.py
 2013-08-21 15:34:31 -07:00
Jey Kottalam 4d737b6d32 Example should make sense 2013-08-21 15:03:37 -07:00
Jey Kottalam 6585f49841 Update build docs 2013-08-21 14:51:56 -07:00
Jey Kottalam 66e7a38a32 Allow build configuration to be set in conf/spark-env.sh 2013-08-21 14:25:53 -07:00
Mark Hamstra ff6f1b0500 Synced sbt and maven builds 2013-08-21 13:50:24 -07:00
Jey Kottalam 31644a011c Use "hadoop.version" property when specifying Hadoop YARN version too 2013-08-21 13:24:28 -07:00
Jey Kottalam 9c6f8df30f Update jekyll plugin to match docs/README.md 2013-08-21 12:57:56 -07:00
Matei Zaharia 111b2741fd Change default SPARK_HADOOP_VERSION in make-distribution.sh too 2013-08-21 11:54:10 -07:00
Reynold Xin 8e3ea4c7db Merge branch 'master' of github.com:mesos/spark 2013-08-21 11:38:51 -07:00
Reynold Xin af602ba9d3 Downgraded default build hadoop version to 1.0.4. 2013-08-21 11:38:24 -07:00
Matei Zaharia 53b1c30607 Update docs for Spark UI port 2013-08-20 22:57:11 -07:00
Patrick Wendell 51a1a0c602 Bump spark version 2013-08-20 22:14:52 -07:00
Reynold Xin 2905611c13 Merge pull request #851 from markhamstra/MutablePairTE 
Removed meaningless types
 2013-08-20 17:36:14 -07:00
Mark Hamstra 5eea613ec0 Removed meaningless types 2013-08-20 16:49:18 -07:00
Ali Ghodsi f20ed14e87 Merged in from upstream to use TaskLocation instead of strings 2013-08-20 16:21:43 -07:00
Ali Ghodsi 5cd21c4195 added curly braces to make the code more consistent 2013-08-20 16:16:05 -07:00
Ali Ghodsi db4bc55bef indent 2013-08-20 16:16:05 -07:00
Ali Ghodsi c0942a710f Bug in test fixed 2013-08-20 16:16:05 -07:00
Ali Ghodsi 5db41919b5 Added a test to make sure no locality preferences are ignored 2013-08-20 16:16:05 -07:00
Ali Ghodsi 7b123b3126 Simpler code 2013-08-20 16:16:05 -07:00
Ali Ghodsi 9192c358e4 simpler code 2013-08-20 16:16:05 -07:00
Ali Ghodsi a75a64eade Fixed almost all of Matei's feedback 2013-08-20 16:16:05 -07:00
Ali Ghodsi f1c853d76d fixed Matei's comments 2013-08-20 16:16:04 -07:00
Ali Ghodsi 890ea6ba79 making CoalescedRDDPartition public 2013-08-20 16:16:04 -07:00
Ali Ghodsi d6b6c680be comment in the test to make it more understandable 2013-08-20 16:16:04 -07:00
Ali Ghodsi b69e7166ba Coalescer now uses current preferred locations for derived RDDs. Made run() in DAGScheduler thread safe and added a method to be able to ask it for preferred locations. Added a similar method that wraps the former inside SparkContext. 2013-08-20 16:16:04 -07:00
Ali Ghodsi 3b5bb8a4ae added one test that will test a future functionality 2013-08-20 16:13:37 -07:00
Ali Ghodsi 33a0f59354 Added error messages to the tests to make failed tests less cryptic 2013-08-20 16:13:37 -07:00