13fd272cd3
### What changes were proposed in this pull request?
This PR intends to fix typos in the sub-modules:
* `R`
* `common`
* `dev`
* `mlib`
* `external`
* `project`
* `streaming`
* `resource-managers`
* `python`
Split per srowen https://github.com/apache/spark/pull/30323#issuecomment-728981618
NOTE: The misspellings have been reported at 706a726f87 (commitcomment-44064356)
### Why are the changes needed?
Misspelled words make it harder to read / understand content.
### Does this PR introduce _any_ user-facing change?
There are various fixes to documentation, etc...
### How was this patch tested?
No testing was performed
Closes #30402 from jsoref/spelling-R_common_dev_mlib_external_project_streaming_resource-managers_python.
Authored-by: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: Sean Owen <srowen@gmail.com>
110 lines
3.9 KiB
Markdown
110 lines
3.9 KiB
Markdown
---
|
|
license: |
|
|
Licensed to the Apache Software Foundation (ASF) under one or more
|
|
contributor license agreements. See the NOTICE file distributed with
|
|
this work for additional information regarding copyright ownership.
|
|
The ASF licenses this file to You under the Apache License, Version 2.0
|
|
(the "License"); you may not use this file except in compliance with
|
|
the License. You may obtain a copy of the License at
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
Unless required by applicable law or agreed to in writing, software
|
|
distributed under the License is distributed on an "AS IS" BASIS,
|
|
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
See the License for the specific language governing permissions and
|
|
limitations under the License.
|
|
---
|
|
|
|
# SparkR CRAN Release
|
|
|
|
To release SparkR as a package to CRAN, we would use the `devtools` package. Please work with the
|
|
`dev@spark.apache.org` community and R package maintainer on this.
|
|
|
|
### Release
|
|
|
|
First, check that the `Version:` field in the `pkg/DESCRIPTION` file is updated. Also, check for stale files not under source control.
|
|
|
|
Note that while `run-tests.sh` runs `check-cran.sh` (which runs `R CMD check`), it is doing so with `--no-manual --no-vignettes`, which skips a few vignettes or PDF checks - therefore it will be preferred to run `R CMD check` on the source package built manually before uploading a release. Also note that for CRAN checks for pdf vignettes to success, `qpdf` tool must be there (to install it, e.g. `yum -q -y install qpdf`).
|
|
|
|
To upload a release, we would need to update the `cran-comments.md`. This should generally contain the results from running the `check-cran.sh` script along with comments on status of all `WARNING` (should not be any) or `NOTE`. As a part of `check-cran.sh` and the release process, the vignettes is build - make sure `SPARK_HOME` is set and Spark jars are accessible.
|
|
|
|
Once everything is in place, run in R under the `SPARK_HOME/R` directory:
|
|
|
|
```R
|
|
paths <- .libPaths(); .libPaths(c("lib", paths)); Sys.setenv(SPARK_HOME=tools::file_path_as_absolute("..")); devtools::release(); .libPaths(paths)
|
|
```
|
|
|
|
For more information please refer to http://r-pkgs.had.co.nz/release.html#release-check
|
|
|
|
### Testing: build package manually
|
|
|
|
To build package manually such as to inspect the resulting `.tar.gz` file content, we would also use the `devtools` package.
|
|
|
|
Source package is what get released to CRAN. CRAN would then build platform-specific binary packages from the source package.
|
|
|
|
#### Build source package
|
|
|
|
To build source package locally without releasing to CRAN, run in R under the `SPARK_HOME/R` directory:
|
|
|
|
```R
|
|
paths <- .libPaths(); .libPaths(c("lib", paths)); Sys.setenv(SPARK_HOME=tools::file_path_as_absolute("..")); devtools::build("pkg"); .libPaths(paths)
|
|
```
|
|
|
|
(http://r-pkgs.had.co.nz/vignettes.html#vignette-workflow-2)
|
|
|
|
Similarly, the source package is also created by `check-cran.sh` with `R CMD build pkg`.
|
|
|
|
For example, this should be the content of the source package:
|
|
|
|
```sh
|
|
DESCRIPTION R inst tests
|
|
NAMESPACE build man vignettes
|
|
|
|
inst/doc/
|
|
sparkr-vignettes.html
|
|
sparkr-vignettes.Rmd
|
|
sparkr-vignettes.Rman
|
|
|
|
build/
|
|
vignette.rds
|
|
|
|
man/
|
|
*.Rd files...
|
|
|
|
vignettes/
|
|
sparkr-vignettes.Rmd
|
|
```
|
|
|
|
#### Test source package
|
|
|
|
To install, run this:
|
|
|
|
```sh
|
|
R CMD INSTALL SparkR_2.1.0.tar.gz
|
|
```
|
|
|
|
With "2.1.0" replaced with the version of SparkR.
|
|
|
|
This command installs SparkR to the default libPaths. Once that is done, you should be able to start R and run:
|
|
|
|
```R
|
|
library(SparkR)
|
|
vignette("sparkr-vignettes", package="SparkR")
|
|
```
|
|
|
|
#### Build binary package
|
|
|
|
To build binary package locally, run in R under the `SPARK_HOME/R` directory:
|
|
|
|
```R
|
|
paths <- .libPaths(); .libPaths(c("lib", paths)); Sys.setenv(SPARK_HOME=tools::file_path_as_absolute("..")); devtools::build("pkg", binary = TRUE); .libPaths(paths)
|
|
```
|
|
|
|
For example, this should be the content of the binary package:
|
|
|
|
```sh
|
|
DESCRIPTION Meta R html tests
|
|
INDEX NAMESPACE help profile worker
|
|
```
|