ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
1422 commits 6 branches 181 tags 397 MiB
Commit graph

1422 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia f855e4fad2 Merge pull request #208 from rxin/dev 
Separated ShuffledRDD into multiple classes.
 2012-09-24 12:32:01 -07:00
root 107a5ca879 Make default number of parallel fetches slightly smaller since it doesn't seem to hurt performance much and it will cause slightly less GC. 2012-09-23 06:06:12 +00:00
root e41cab04ca Avoid creating an extra buffer when saving a stream of values as DISK_ONLY 2012-09-23 05:56:44 +00:00
Matei Zaharia 33fb373e69 Merge pull request #215 from dennybritz/dev 
HTTP File server fixes
 2012-09-21 17:39:28 -07:00
Denny afb7ccc838 HTTP File server fixes. 2012-09-21 10:58:13 -07:00
root 6d28dde370 Rename our toIterator method into asIterator to prevent confusion with the 
Scala collection one, which often *copies* a collection.
 2012-09-21 06:02:55 +00:00
root a642051ade Fixed a performance bug in BlockManager that was creating garbage when 
returning deserialized, in-memory RDDs.
 2012-09-21 05:42:21 +00:00
root 8feb5caacd Fixed an issue with ordering of classloader setup that was causing Java deserializer to break 2012-09-21 05:13:19 +00:00
Matei Zaharia bf891a5c18 Merge pull request #203 from JoshRosen/docs/java-programming-guide 
Java Programming Guide
 2012-09-20 17:35:10 -07:00
Matei Zaharia 1c3d3f4e8c Merge pull request #210 from rxin/deploy-limit-retrycount 
Set a limited number of retry in standalone deploy mode.
 2012-09-20 17:33:47 -07:00
Reynold Xin 6b5980da79 Set a limited number of retry in standalone deploy mode. 2012-09-19 15:41:56 -07:00
Reynold Xin 397d3816e1 Separated ShuffledRDD into multiple classes: RepartitionShuffledRDD, 
ShuffledSortedRDD, and ShuffledAggregatedRDD.
 2012-09-19 12:31:45 -07:00
Matei Zaharia 9a449e0063 Merge pull request #204 from dennybritz/dev 
When a file is downloaded, make it executable.
 2012-09-17 10:11:22 -07:00
Denny ca64d16a2d When a file is downloaded, make it executable. That's neccsary for scripts (e.g. in Shark) 2012-09-17 10:08:37 -07:00
Josh Rosen c94e9cc54a Add Java Programming Guide; fix broken doc links. 2012-09-16 20:46:46 -07:00
Matei Zaharia 098ae55db1 Merge pull request #202 from andyk/doc 
Updates to docs (including nav structure)
 2012-09-16 17:18:38 -07:00
Andy Konwinski 8f7dfcf332 Merge remote-tracking branch 'public-spark/dev' into doc 2012-09-16 15:31:30 -07:00
Andy Konwinski a92748d3ba Merge remote-tracking branch 'public-spark/dev' into doc 2012-09-16 15:29:35 -07:00
Andy Konwinski 52c29071a4 - Add docs/api to .gitignore 
- Rework/expand the nav bar with more of the docs site
- Removing parts of docs about EC2 and Mesos that differentiate between
  running 0.5 and before
    - Merged subheadings from running-on-amazon-ec2.html that are still relevant
      (i.e., "Using a newer version of Spark" and "Accessing Data in S3") into
      ec2-scripts.html and deleted running-on-amazon-ec2.html
- Added some TODO comments to a few docs
- Updated the blurb about AMP Camp
- Renamed programming-guide to spark-programming-guide
- Fixing typos/etc. in Standalone Spark doc
 2012-09-16 15:28:52 -07:00
Matei Zaharia bcd7b40578 Merge pull request #200 from andyk/doc 
Updates to docs
 2012-09-13 18:10:19 -07:00
Matei Zaharia 840cbcf849 Change default serializer to Java.. it had accidentally become Kryo. 2012-09-13 17:19:26 -07:00
Andy Konwinski 6765d9727e Adds a jekyll plugin (written in Ruby) to the _plugins directory 
which generates scala doc by calling `sbt/sbt doc`, copies it over
to docs, and updates the links from the api webpage to now point to
the copied over scaladoc (making the _site directory easy to just
copy over to a public website).
 2012-09-13 17:17:58 -07:00
Andy Konwinski 462cd8be60 Re-enabling responsive for better looking padding and more sane resizing, 
but removed the collapsable stuff from the nav bar.
 2012-09-13 15:39:35 -07:00
Andy Konwinski 5ec7a6665b More crisp logo created from vector source (ai) and disabled 
responsive css (so nav menu doesn't switch to collapsed version
for narrow viewports).
 2012-09-13 15:27:33 -07:00
Andy Konwinski b0207e2bfd Replaces "Spark" word in nav bar with logo. 2012-09-13 12:08:12 -07:00
Andy Konwinski 130f2f2f41 Merge remote-tracking branch 'public-spark/dev' into doc 2012-09-13 11:29:25 -07:00
Matei Zaharia 77a0ee99fa Merge pull request #199 from dennybritz/dev 
YARN and standalone documentation
 2012-09-13 10:02:04 -07:00
Denny 638a8511fa Fixed navbar style problem 2012-09-13 09:56:27 -07:00
Denny 6d53b971b9 Added standalone and YARN docs. Merged standalone cluster into standalone doc 2012-09-13 09:47:54 -07:00
Andy Konwinski ca2c999e0f Making the link to api scaladocs work and migrating other code snippets 
to use pygments syntax highlighting.
 2012-09-12 23:25:07 -07:00
Andy Konwinski c4db09ea76 Adds ec2-scripts.md back (it was mistakenly removed earlier due to 
git weirdness).
 2012-09-12 20:56:32 -07:00
Matei Zaharia d3db46fdef Remove title from content in Bagel 2012-09-12 19:50:38 -07:00
Matei Zaharia 88181bee9e Small tweaks to generated doc pages 2012-09-12 19:47:31 -07:00
Matei Zaharia 35e17be840 Merge branch 'dev' of github.com:mesos/spark into dev 2012-09-12 19:38:15 -07:00
Matei Zaharia c92e6169cf Merge pull request #198 from andyk/doc 
Fix links and make things a bit prettier.
 2012-09-12 19:37:51 -07:00
Andy Konwinski 1bcd09e093 Merge remote-tracking branch 'public-spark/dev' into doc 2012-09-12 19:28:55 -07:00
Andy Konwinski 35adccd008 Adds syntax highlighting (via pygments), and some style tweaks to make things 
easier to read.
 2012-09-12 19:27:44 -07:00
Reynold Xin 4b8b0d5c08 Added a link to AMPCamp website in Spark docs. 2012-09-12 17:54:16 -07:00
Reynold Xin b603e55a0a Changed block storage level for shuffle to DISK_ONLY. 2012-09-12 17:54:01 -07:00
Andy Konwinski bf54ad2fe0 Merge remote-tracking branch 'public-spark/dev' into doc 2012-09-12 16:36:43 -07:00
Andy Konwinski 95c5a376b5 Fixing a hanging sentence in docs/ec2-scripts.md 2012-09-12 16:17:24 -07:00
Andy Konwinski 3c7e40fa27 Removed the upper-case version of docs/EC2-Scripts.md. 2012-09-12 16:09:34 -07:00
Andy Konwinski 4d3a17c8d7 Fixing lots of broken links. 2012-09-12 16:06:18 -07:00
Matei Zaharia b4dfa25c8a Store shuffle map outputs as DISK_ONLY 2012-09-12 16:05:57 -07:00
Matei Zaharia 2d761e3353 Ported performance and FT improvements from latest streaming work 2012-09-12 14:54:40 -07:00
Matei Zaharia 9b4cd1648b Fix bugs with Connection's shutdown callback failing to get its address 2012-09-12 14:54:14 -07:00
Matei Zaharia 9199775d41 Wait for Akka to really shut down in SparkEnv.stop() 2012-09-12 14:50:37 -07:00
Matei Zaharia d5bdd12353 Merge pull request #197 from andyk/doc 
Adds a "docs" directory containing existing Spark documentation and doc build instructions
 2012-09-12 13:50:50 -07:00
Andy Konwinski 49e98500a9 Updated base README to point to documentation site instead of wiki, updated 
docs/README.md to describe use of Jekyll, and renmaed things to make them
more consistent with the lower-case-with-hyphens convention.
 2012-09-12 13:03:43 -07:00
Andy Konwinski 16da942d66 Adding docs directory containing documentation currently on the wiki 
which can be compiled via jekyll, using the command `jekyll`. To compile
and run a local webserver to serve the doc as a website, run
`jekyll --server`.
 2012-09-12 13:03:43 -07:00