### What changes were proposed in this pull request?
This PR intends to fix typos in the sub-modules:
* `R`
* `common`
* `dev`
* `mlib`
* `external`
* `project`
* `streaming`
* `resource-managers`
* `python`
Split per srowen https://github.com/apache/spark/pull/30323#issuecomment-728981618
NOTE: The misspellings have been reported at 706a726f87 (commitcomment-44064356)
### Why are the changes needed?
Misspelled words make it harder to read / understand content.
### Does this PR introduce _any_ user-facing change?
There are various fixes to documentation, etc...
### How was this patch tested?
No testing was performed
Closes #30402 from jsoref/spelling-R_common_dev_mlib_external_project_streaming_resource-managers_python.
Authored-by: Josh Soref <jsoref@users.noreply.github.com>
Signed-off-by: Sean Owen <srowen@gmail.com>
3.9 KiB
license |
---|
Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. |
SparkR CRAN Release
To release SparkR as a package to CRAN, we would use the devtools
package. Please work with the
dev@spark.apache.org
community and R package maintainer on this.
Release
First, check that the Version:
field in the pkg/DESCRIPTION
file is updated. Also, check for stale files not under source control.
Note that while run-tests.sh
runs check-cran.sh
(which runs R CMD check
), it is doing so with --no-manual --no-vignettes
, which skips a few vignettes or PDF checks - therefore it will be preferred to run R CMD check
on the source package built manually before uploading a release. Also note that for CRAN checks for pdf vignettes to success, qpdf
tool must be there (to install it, e.g. yum -q -y install qpdf
).
To upload a release, we would need to update the cran-comments.md
. This should generally contain the results from running the check-cran.sh
script along with comments on status of all WARNING
(should not be any) or NOTE
. As a part of check-cran.sh
and the release process, the vignettes is build - make sure SPARK_HOME
is set and Spark jars are accessible.
Once everything is in place, run in R under the SPARK_HOME/R
directory:
paths <- .libPaths(); .libPaths(c("lib", paths)); Sys.setenv(SPARK_HOME=tools::file_path_as_absolute("..")); devtools::release(); .libPaths(paths)
For more information please refer to http://r-pkgs.had.co.nz/release.html#release-check
Testing: build package manually
To build package manually such as to inspect the resulting .tar.gz
file content, we would also use the devtools
package.
Source package is what get released to CRAN. CRAN would then build platform-specific binary packages from the source package.
Build source package
To build source package locally without releasing to CRAN, run in R under the SPARK_HOME/R
directory:
paths <- .libPaths(); .libPaths(c("lib", paths)); Sys.setenv(SPARK_HOME=tools::file_path_as_absolute("..")); devtools::build("pkg"); .libPaths(paths)
(http://r-pkgs.had.co.nz/vignettes.html#vignette-workflow-2)
Similarly, the source package is also created by check-cran.sh
with R CMD build pkg
.
For example, this should be the content of the source package:
DESCRIPTION R inst tests
NAMESPACE build man vignettes
inst/doc/
sparkr-vignettes.html
sparkr-vignettes.Rmd
sparkr-vignettes.Rman
build/
vignette.rds
man/
*.Rd files...
vignettes/
sparkr-vignettes.Rmd
Test source package
To install, run this:
R CMD INSTALL SparkR_2.1.0.tar.gz
With "2.1.0" replaced with the version of SparkR.
This command installs SparkR to the default libPaths. Once that is done, you should be able to start R and run:
library(SparkR)
vignette("sparkr-vignettes", package="SparkR")
Build binary package
To build binary package locally, run in R under the SPARK_HOME/R
directory:
paths <- .libPaths(); .libPaths(c("lib", paths)); Sys.setenv(SPARK_HOME=tools::file_path_as_absolute("..")); devtools::build("pkg", binary = TRUE); .libPaths(paths)
For example, this should be the content of the binary package:
DESCRIPTION Meta R html tests
INDEX NAMESPACE help profile worker