Matei Zaharia
682b2d9329
Added a test for when an RDD only partially fits in memory
2012-10-12 14:58:26 -07:00
Matei Zaharia
dca496bb77
Document cartesian() operation
2012-10-12 14:46:41 -07:00
Matei Zaharia
1183b30941
Merge branch 'dev' of github.com:mesos/spark into dev
2012-10-12 14:40:07 -07:00
Matei Zaharia
603b419fdf
Tweak
2012-10-12 14:40:00 -07:00
Matei Zaharia
23015ccac0
Merge pull request #271 from shivaram/block-manager-npe-fix
...
Change block manager to accept a ArrayBuffer
2012-10-12 14:36:28 -07:00
Shivaram Venkataraman
8577523f37
Add test to verify if RDD is computed even if block manager has insufficient
...
memory
2012-10-12 14:14:57 -07:00
Matei Zaharia
bd78bbb2cf
Merge pull request #270 from pwendell/java-javadoc
...
Adding Java documentation
2012-10-11 12:21:47 -07:00
Patrick Wendell
dc8adbd359
Adding Java documentation
2012-10-11 00:49:03 -07:00
Shivaram Venkataraman
2cf40c5fd5
Change block manager to accept a ArrayBuffer instead of an iterator to ensure
...
that the computation can proceed even if we run out of memory to cache the
block. Update CacheTracker to use this new interface
2012-10-11 00:42:46 -07:00
Matei Zaharia
cce56835cd
Comment out Sonatype publishing stuff so publish-local works
2012-10-10 22:11:31 -07:00
Matei Zaharia
4001cbdec1
Merge pull request #268 from pwendell/sonatype
...
Adding code for publishing to Sonatype.
2012-10-10 18:57:32 -07:00
Patrick Wendell
6d328f54d0
Changing tabs to spaces
2012-10-10 18:54:22 -07:00
Matei Zaharia
066e979e9e
Merge pull request #267 from dennybritz/dev
...
Fixed bug when fetching Jar dependencies.
2012-10-10 18:32:23 -07:00
Patrick Wendell
3ed172ea59
Adding code for publishing to Sonatype.
...
By default - I'm leaving this commented out. This is because
there is a bug in the PGP signing plugin which causes it to active
even duing a publish-local. So we'll just uncomment when we decide
to publish.
2012-10-10 17:25:29 -07:00
Denny
d3f095f904
Fixed bug when fetching Jar dependencies.
...
Instead of checking currentFiles check currentJars.
2012-10-10 16:09:53 -07:00
Matei Zaharia
1ac7bf893b
Merge pull request #265 from pwendell/docs-version-config
...
Making Spark version configurable in docs and updating Bagel doc
2012-10-10 11:02:29 -07:00
Patrick Wendell
5788a92953
Updating Bagel build instructions
2012-10-09 22:39:29 -07:00
Patrick Wendell
0f760a0bd3
Updating programming guide with new link instructions
2012-10-09 22:39:29 -07:00
Patrick Wendell
4de5cc1ad4
Removing reference to publish-local in the quickstart
2012-10-09 22:39:29 -07:00
Patrick Wendell
8321e7f0c2
Fixing YARN instructions
2012-10-09 22:39:28 -07:00
Patrick Wendell
5013c785fd
Adding SNAPSHOT to Spark version in doc config
2012-10-09 22:39:28 -07:00
Matei Zaharia
ee2fcb2ce6
Added documentation to all the *RDDFunction classes, and moved them into
...
the spark package to make them more visible. Also documented various
other miscellaneous things in the API.
2012-10-09 18:38:36 -07:00
Matei Zaharia
95a435cdd4
Increase version on 0.5 branch to 0.5.2-SNAPSHOT
2012-10-09 18:17:51 -07:00
Matei Zaharia
bc0bc672d0
Updates to documentation:
...
- Edited quick start and tuning guide to simplify them a little
- Simplified top menu bar
- Made private a SparkContext constructor parameter that was left as
public
- Various small fixes
2012-10-09 14:30:23 -07:00
Matei Zaharia
ad28aebb0a
Merge pull request #262 from andyk/document-public-apis
...
Document RDD api (i.e. RDD.scala)
2012-10-09 09:47:58 -07:00
Matei Zaharia
4780fee887
Merge pull request #260 from andyk/update-docs-to-use-version-vars
...
Updates docs to use the new version num vars and adds Spark version in nav bar
2012-10-09 09:46:20 -07:00
Matei Zaharia
d3b8252050
Merge pull request #263 from RayRacine/m1.medium
...
Add m1.medium node option to cluster management script.
2012-10-09 09:44:58 -07:00
Ray Racine
c52bd1b6b1
Add m1.medium node option to cluster management script.
2012-10-09 11:05:13 -04:00
Andy Konwinski
8aec63b0bc
Merge remote-tracking branch 'public-spark/dev' into document-public-apis
2012-10-08 22:50:58 -07:00
Andy Konwinski
1d79ff6028
Fixes a typo, adds scaladoc comments to SparkContext constructors.
2012-10-08 22:49:17 -07:00
Patrick Wendell
ac310098ef
More docs in RDD class
2012-10-08 22:25:11 -07:00
Andy Konwinski
bd688940a1
A start on scaladoc for the public APIs.
2012-10-08 21:13:29 -07:00
Matei Zaharia
a91ed6a3a7
Merge pull request #256 from pwendell/new-download-instr
...
Adding new download instructions
2012-10-08 17:25:25 -07:00
Andy Konwinski
e1a724f39c
Updating lots of docs to use the new special version number variables,
...
also adding the version to the navbar so it is easy to tell which
version of Spark these docs were compiled for.
2012-10-08 17:17:17 -07:00
Patrick Wendell
7887e171f4
Adding new download instructions
2012-10-08 17:05:13 -07:00
Matei Zaharia
1231eb12e6
Merge pull request #259 from mosharaf/bc-fix-dev
...
Synchronization bug fix in broadcast implementations
2012-10-08 16:54:28 -07:00
Mosharaf Chowdhury
edc67bfba8
Merge branch 'dev' into bc-fix-dev
2012-10-08 16:19:13 -07:00
Matei Zaharia
4f1e0325f1
Merge pull request #257 from andyk/add-version-vars-to-docs
...
Adds special version variables to docs templating system
2012-10-08 12:29:08 -07:00
Andy Konwinski
56b91ab380
Updates README.md with instructions for running jekyll without building
...
scaladoc (i.e. run `SKIP_SCALADOC=1 jekyll`).
2012-10-08 12:15:44 -07:00
Andy Konwinski
89f8e1c2e7
Merge remote-tracking branch 'public-spark/dev' into add-version-vars-to-docs
...
Conflicts:
docs/quick-start.md
2012-10-08 12:15:38 -07:00
Matei Zaharia
88152e2164
Merge pull request #254 from pwendell/quickstart-fix
...
Removing one link in quickstart
2012-10-08 11:08:29 -07:00
Andy Konwinski
45d03231d0
Adds liquid variables to docs templating system so that they can be used
...
throughout the docs: SPARK_VERSION, SCALA_VERSION, and MESOS_VERSION.
To use them, e.g. use {{site.SPARK_VERSION}}.
Also removes uses of {{HOME_PATH}} which were being resolved to ""
by the templating system anyway.
2012-10-08 10:30:38 -07:00
Mosharaf Chowdhury
46c3899836
Merge remote-tracking branch 'upstream/dev' into dev
2012-10-08 09:47:46 -07:00
Matei Zaharia
280286b6cd
Merge pull request #255 from andyk/fix-nav-gap
...
Removes the annoying small gap above the nav menu dropdown boxes
2012-10-08 09:45:03 -07:00
Andy Konwinski
cd710a7718
Fixes the small gap above the nav menu dropdown boxes and the hoverable
...
menu items that was causing the dropdowns to go away when the user
moved their mouse down towards them.
2012-10-08 09:33:40 -07:00
Patrick Wendell
99aac2f6c4
Removing one link in quickstart
2012-10-08 08:53:27 -07:00
Matei Zaharia
efc5423210
Made compression configurable separately for shuffle, broadcast and RDDs
2012-10-07 11:30:53 -07:00
Matei Zaharia
d1538ebdd9
Change version in REPL
2012-10-07 10:40:29 -07:00
Matei Zaharia
f31b6f92b0
Changes to run script and README to deal with updated Scala version and
...
with Mesos being in Maven
2012-10-07 10:00:48 -07:00
Matei Zaharia
039cc6228e
Merge pull request #251 from JoshRosen/docs/internals
...
Document Dependency classes and make minor interface improvements
2012-10-07 09:56:53 -07:00