ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
2626 commits 6 branches 181 tags 397 MiB
Commit graph

2626 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia 65113b7e1b Only group elements ten at a time into SequenceFile records in 
saveAsObjectFile
 2012-10-06 17:14:41 -07:00
Matei Zaharia dbf1f3dd5b Make reduce logging less verbose 2012-10-06 17:10:09 -07:00
Matei Zaharia 716e10ca32 Minor formatting fixes 2012-10-05 22:03:06 -07:00
Matei Zaharia 70f02fa912 Merge branch 'dev' of github.com:mesos/spark into dev 2012-10-05 22:00:22 -07:00
Matei Zaharia 95ef307ef5 Merge pull request #249 from andyk/move-rdds-to-their-own-package 
Move RDD classes/files to their own package/directory
 2012-10-05 21:59:57 -07:00
Andy Konwinski a242cdd0a6 Factor subclasses of RDD out of RDD.scala into their own classes 
in the rdd package.
 2012-10-05 19:53:54 -07:00
Andy Konwinski d7363a6b8a Moves all files in core/src/main/scala/ that have RDD in their name 
from that directory to a new core/src/main/scala/rdd directory.
 2012-10-05 19:23:45 -07:00
Andy Konwinski e0067da082 Moves all files in core/src/main/scala/ that have RDD in them from 
package spark to package spark.rdd and updates all references to them.
 2012-10-05 19:23:45 -07:00
Matei Zaharia 69588baf65 Cleaning up code slightly 2012-10-05 19:16:09 -07:00
root f52bc09a34 Reduce some overly aggressive logging in connection manager 2012-10-06 01:54:39 +00:00
Matei Zaharia e6e27a05d8 Links quick start from nav bar 2012-10-05 17:06:55 -07:00
Shivaram Venkataraman b6e4f46a96 Fix SizeEstimator tests to work with String classes in JDK 6 and 7 
Conflicts:

	core/src/test/scala/spark/BoundedMemoryCacheSuite.scala
 2012-10-05 16:58:57 -07:00
Matei Zaharia 7eae2d1229 Merge branch 'master' into reduce-logging 2012-10-05 16:36:16 -07:00
Matei Zaharia d9bcc2a6e3 Merge branch 'master' of github.com:mesos/spark 2012-10-05 16:35:48 -07:00
Matei Zaharia 1620d59ea4 Merge pull request #246 from shivaram/size-estimator-fix-master 
Fix SizeEstimator tests to work with String classes in JDK 6 and 7
 2012-10-05 10:28:30 -07:00
Matei Zaharia e3ae98b54e Merge pull request #247 from squito/dev 
Dev
 2012-10-05 10:27:18 -07:00
Imran Rashid e0698f8f26 change tests to show utility of localValue 2012-10-04 23:05:42 -07:00
Imran Rashid 82a3327862 make accumulator.localValue public, add tests 
Conflicts:
	core/src/test/scala/spark/AccumulatorSuite.scala
 2012-10-04 23:05:01 -07:00
Matei Zaharia 8c82f43db3 Scaladoc documentation for some core Spark functionality 2012-10-04 22:59:36 -07:00
Matei Zaharia 08edb04f47 Merge branch 'dev' of github.com:mesos/spark into dev 2012-10-04 22:11:32 -07:00
Shivaram Venkataraman 5975d2ee3b Fix SizeEstimator tests to work with String classes in JDK 6 and 7 2012-10-04 19:42:57 -07:00
Matei Zaharia 5a7b370225 Only group elements ten at a time into SequenceFile records in 
saveAsObjectFile
 2012-10-04 16:49:30 -07:00
Matei Zaharia 66d7066d4f Let the reducer retry if a fetch fails before reading all records 2012-10-04 16:41:17 -07:00
Matei Zaharia c535762deb Don't check for JARs in core/lib anymore 2012-10-04 15:11:43 -07:00
Matei Zaharia 588120cd71 Add more logging for number of records fetched by each reduce 2012-10-04 11:54:47 -07:00
Matei Zaharia d6d071f19a Merge pull request #245 from pwendell/mem_tuning 
Some additions to the Tuning Guide.
 2012-10-03 22:00:48 -07:00
Patrick Wendell e84c068fab Some additions to the Tuning Guide. 
1. Slight change in organization
2. Added pre-requisites
3. Made a new section about determining memory footprint
   of an RDD
4. Other small changes
 2012-10-03 14:06:34 -07:00
Matei Zaharia c04aeaf365 Merge pull request #243 from andyk/mesos-from-maven-central 
Removes the included mesos-0.9.0.jar and pulls it from Maven Central instead
 2012-10-03 11:10:00 -07:00
Reynold Xin 66d848175a Merge pull request #244 from rxin/dev 
Made Serializer and JavaSerializer non private.
 2012-10-03 11:01:48 -07:00
Reynold Xin 45f4b7cc7e Made Serializer and JavaSerializer non private. 2012-10-03 10:20:59 -07:00
Andy Konwinski 5897567679 Removes the included mesos-0.9.0.jar and adds a libraryDependency to 
the build file so that mesos-0.9.0-incubating.jar (which contains the
same class files, but has a silightly different name) will be pulled
down from Maven Central instead.
 2012-10-03 08:58:05 -07:00
Matei Zaharia 42cd148507 Merge pull request #236 from pwendell/quickstart 
A Spark "Quick Start" example
 2012-10-03 08:31:43 -07:00
Matei Zaharia 833f1d0c86 Made StorageLevel public 2012-10-03 08:27:25 -07:00
Patrick Wendell 35b767f478 Responding to Matei's comments 2012-10-02 23:54:03 -07:00
Matei Zaharia 6cf5dffc72 Make more stuff private[spark] 2012-10-02 22:28:55 -07:00
Mosharaf Chowdhury 119e50c7b9 Conflict fixed 2012-10-02 22:25:39 -07:00
Mosharaf Chowdhury ff813e4380 Merge remote-tracking branch 'upstream/dev' into dev 2012-10-02 22:17:17 -07:00
Matei Zaharia 87f4451f20 Simplify README even further 2012-10-02 22:14:40 -07:00
Matei Zaharia a17e66a689 Merge pull request #242 from pwendell/readme-update 
Changing version of Scala in README
 2012-10-02 22:12:08 -07:00
Patrick Wendell 89be3c3e76 Changing version of Scala in README 2012-10-02 22:10:23 -07:00
Matei Zaharia a3e54de6ca Merge pull request #241 from shivaram/tuning-doc 
First cut at adding documentation for GC tuning
 2012-10-02 21:26:54 -07:00
Matei Zaharia 626f701931 Merge pull request #240 from dennybritz/private_classes 
Package-Private Classes
 2012-10-02 21:24:32 -07:00
Shivaram Venkataraman 3d2b900b08 First cut at adding documentation for GC tuning 2012-10-02 20:07:18 -07:00
Denny 0361353a70 Make Java API abstract wrapped functions private 2012-10-02 20:02:53 -07:00
Denny b9badcd5bd accidentially removed trait 2012-10-02 19:35:07 -07:00
Denny 18a1faedf6 Stylistic changes and Public Accumulable and Broadcast 2012-10-02 19:28:37 -07:00
Denny b7a913e1fa Make dependency classes public - used by spark 2012-10-02 19:04:23 -07:00
Denny 4d9f4b01af Make classes package private 2012-10-02 19:00:19 -07:00
Matei Zaharia 97cbd699d7 Merge branch 'dev' of github.com:mesos/spark into dev 2012-10-02 17:31:01 -07:00
Matei Zaharia 5fda59ab99 Added a test for overly large blocks in memory store 2012-10-02 17:30:40 -07:00