ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
1382 commits 6 branches 181 tags 397 MiB
Commit graph

1382 commits

This branch
This branch
All branches
Author SHA1 Message Date
Matei Zaharia bd78bbb2cf Merge pull request #270 from pwendell/java-javadoc 
Adding Java documentation
 2012-10-11 12:21:47 -07:00
Patrick Wendell dc8adbd359 Adding Java documentation 2012-10-11 00:49:03 -07:00
Shivaram Venkataraman 2cf40c5fd5 Change block manager to accept a ArrayBuffer instead of an iterator to ensure 
that the computation can proceed even if we run out of memory to cache the
block. Update CacheTracker to use this new interface
 2012-10-11 00:42:46 -07:00
Matei Zaharia cce56835cd Comment out Sonatype publishing stuff so publish-local works 2012-10-10 22:11:31 -07:00
Matei Zaharia 4001cbdec1 Merge pull request #268 from pwendell/sonatype 
Adding code for publishing to Sonatype.
 2012-10-10 18:57:32 -07:00
Patrick Wendell 6d328f54d0 Changing tabs to spaces 2012-10-10 18:54:22 -07:00
Matei Zaharia 066e979e9e Merge pull request #267 from dennybritz/dev 
Fixed bug when fetching Jar dependencies.
 2012-10-10 18:32:23 -07:00
Patrick Wendell 3ed172ea59 Adding code for publishing to Sonatype. 
By default - I'm leaving this commented out. This is because
there is a bug in the PGP signing plugin which causes it to active
even duing a publish-local. So we'll just uncomment when we decide
to publish.
 2012-10-10 17:25:29 -07:00
Denny d3f095f904 Fixed bug when fetching Jar dependencies. 
Instead of checking currentFiles check currentJars.
 2012-10-10 16:09:53 -07:00
Matei Zaharia 1ac7bf893b Merge pull request #265 from pwendell/docs-version-config 
Making Spark version configurable in docs and updating Bagel doc
 2012-10-10 11:02:29 -07:00
Patrick Wendell 5788a92953 Updating Bagel build instructions 2012-10-09 22:39:29 -07:00
Patrick Wendell 0f760a0bd3 Updating programming guide with new link instructions 2012-10-09 22:39:29 -07:00
Patrick Wendell 4de5cc1ad4 Removing reference to publish-local in the quickstart 2012-10-09 22:39:29 -07:00
Patrick Wendell 8321e7f0c2 Fixing YARN instructions 2012-10-09 22:39:28 -07:00
Patrick Wendell 5013c785fd Adding SNAPSHOT to Spark version in doc config 2012-10-09 22:39:28 -07:00
Matei Zaharia ee2fcb2ce6 Added documentation to all the *RDDFunction classes, and moved them into 
the spark package to make them more visible. Also documented various
other miscellaneous things in the API.
 2012-10-09 18:38:36 -07:00
Matei Zaharia 95a435cdd4 Increase version on 0.5 branch to 0.5.2-SNAPSHOT 2012-10-09 18:17:51 -07:00
Matei Zaharia bc0bc672d0 Updates to documentation: 
- Edited quick start and tuning guide to simplify them a little
- Simplified top menu bar
- Made private a SparkContext constructor parameter that was left as
  public
- Various small fixes
 2012-10-09 14:30:23 -07:00
Matei Zaharia ad28aebb0a Merge pull request #262 from andyk/document-public-apis 
Document RDD api (i.e. RDD.scala)
 2012-10-09 09:47:58 -07:00
Matei Zaharia 4780fee887 Merge pull request #260 from andyk/update-docs-to-use-version-vars 
Updates docs to use the new version num vars and adds Spark version in nav bar
 2012-10-09 09:46:20 -07:00
Matei Zaharia d3b8252050 Merge pull request #263 from RayRacine/m1.medium 
Add m1.medium node option to cluster management script.
 2012-10-09 09:44:58 -07:00
Ray Racine c52bd1b6b1 Add m1.medium node option to cluster management script. 2012-10-09 11:05:13 -04:00
Andy Konwinski 8aec63b0bc Merge remote-tracking branch 'public-spark/dev' into document-public-apis 2012-10-08 22:50:58 -07:00
Andy Konwinski 1d79ff6028 Fixes a typo, adds scaladoc comments to SparkContext constructors. 2012-10-08 22:49:17 -07:00
Patrick Wendell ac310098ef More docs in RDD class 2012-10-08 22:25:11 -07:00
Andy Konwinski bd688940a1 A start on scaladoc for the public APIs. 2012-10-08 21:13:29 -07:00
Matei Zaharia a91ed6a3a7 Merge pull request #256 from pwendell/new-download-instr 
Adding new download instructions
 2012-10-08 17:25:25 -07:00
Andy Konwinski e1a724f39c Updating lots of docs to use the new special version number variables, 
also adding the version to the navbar so it is easy to tell which
version of Spark these docs were compiled for.
 2012-10-08 17:17:17 -07:00
Patrick Wendell 7887e171f4 Adding new download instructions 2012-10-08 17:05:13 -07:00
Matei Zaharia 1231eb12e6 Merge pull request #259 from mosharaf/bc-fix-dev 
Synchronization bug fix in broadcast implementations
 2012-10-08 16:54:28 -07:00
Mosharaf Chowdhury edc67bfba8 Merge branch 'dev' into bc-fix-dev 2012-10-08 16:19:13 -07:00
Matei Zaharia 4f1e0325f1 Merge pull request #257 from andyk/add-version-vars-to-docs 
Adds special version variables to docs templating system
 2012-10-08 12:29:08 -07:00
Andy Konwinski 56b91ab380 Updates README.md with instructions for running jekyll without building 
scaladoc (i.e. run `SKIP_SCALADOC=1 jekyll`).
 2012-10-08 12:15:44 -07:00
Andy Konwinski 89f8e1c2e7 Merge remote-tracking branch 'public-spark/dev' into add-version-vars-to-docs 
Conflicts:
	docs/quick-start.md
 2012-10-08 12:15:38 -07:00
Matei Zaharia 88152e2164 Merge pull request #254 from pwendell/quickstart-fix 
Removing one link in quickstart
 2012-10-08 11:08:29 -07:00
Andy Konwinski 45d03231d0 Adds liquid variables to docs templating system so that they can be used 
throughout the docs: SPARK_VERSION, SCALA_VERSION, and MESOS_VERSION.

To use them, e.g. use {{site.SPARK_VERSION}}.

Also removes uses of {{HOME_PATH}} which were being resolved to ""
by the templating system anyway.
 2012-10-08 10:30:38 -07:00
Mosharaf Chowdhury 46c3899836 Merge remote-tracking branch 'upstream/dev' into dev 2012-10-08 09:47:46 -07:00
Matei Zaharia 280286b6cd Merge pull request #255 from andyk/fix-nav-gap 
Removes the annoying small gap above the nav menu dropdown boxes
 2012-10-08 09:45:03 -07:00
Andy Konwinski cd710a7718 Fixes the small gap above the nav menu dropdown boxes and the hoverable 
menu items that was causing the dropdowns to go away when the user
moved their mouse down towards them.
 2012-10-08 09:33:40 -07:00
Patrick Wendell 99aac2f6c4 Removing one link in quickstart 2012-10-08 08:53:27 -07:00
Matei Zaharia efc5423210 Made compression configurable separately for shuffle, broadcast and RDDs 2012-10-07 11:30:53 -07:00
Matei Zaharia d1538ebdd9 Change version in REPL 2012-10-07 10:40:29 -07:00
Matei Zaharia f31b6f92b0 Changes to run script and README to deal with updated Scala version and 
with Mesos being in Maven
 2012-10-07 10:00:48 -07:00
Matei Zaharia 039cc6228e Merge pull request #251 from JoshRosen/docs/internals 
Document Dependency classes and make minor interface improvements
 2012-10-07 09:56:53 -07:00
Matei Zaharia b97bde2b62 Merge pull request #252 from pwendell/0.5-with-sonatype 
Adding Sonatype releases to SBT.
 2012-10-07 09:47:46 -07:00
Reynold Xin 933e4f5123 Merge pull request #253 from rxin/dev 
Changed the println to logInfo in Utils.fetchFile.
 2012-10-07 01:54:18 -07:00
Reynold Xin f66c0e9561 Changed the println to logInfo in Utils.fetchFile. 2012-10-07 01:53:24 -07:00
Patrick Wendell 14719b93ff Adding Sonatype releases to SBT. 
This does a few things to get this branch ready for release:

1. Upgrades the sbt and Scala version
2. Sets the release number to 0.5.1
3. Adds the Sonatype publishing target
4. Installs the PGP signing plugin
5. Removes the Mesos jar dependency
 2012-10-07 01:20:39 -07:00
Matei Zaharia d72db3d7dc Merge pull request #250 from rxin/dev 
Fixed a bug in addFile that if the file is specified as "file:///", the symlink is created incorrectly for local mode.
 2012-10-07 00:56:53 -07:00
Reynold Xin 80f59e17e2 Fixed a bug in addFile that if the file is specified as "file:///", the 
symlink is created wrong for local mode.
 2012-10-07 00:54:38 -07:00