7892f88f84
### What changes were proposed in this pull request? This PR makes the following refinements to the workflow for building docs: * Install Python and Ruby consistently using pyenv and rbenv across both the docs README and the release Dockerfile. * Pin the Python and Ruby versions we use. * Pin all direct Python and Ruby dependency versions. * Eliminate any use of `sudo pip`, which the Python community discourages, or `sudo gem`. ### Why are the changes needed? This PR should increase the consistency and reproducibility of the doc-building process by managing Python and Ruby in a more consistent way, and by eliminating unused or outdated code. Here's a possible example of an issue building the docs that would be addressed by the changes in this PR: https://github.com/apache/spark/pull/27459#discussion_r376135719 ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Manual tests: * I was able to build the Docker image successfully, minus the final part about `RUN useradd`. * I am unable to run `do-release-docker.sh` because I am not a committer and don't have the required GPG key. * I built the docs locally and viewed them in the browser. I think I need a committer to more fully test out these changes. Closes #27534 from nchammas/SPARK-30731-building-docs. Authored-by: Nicholas Chammas <nicholas.chammas@liveramp.com> Signed-off-by: Sean Owen <srowen@gmail.com> |
||
|---|---|---|
| .. | ||
| create-release | ||
| deps | ||
| sparktestsupport | ||
| tests | ||
| .gitignore | ||
| .rat-excludes | ||
| .scalafmt.conf | ||
| appveyor-guide.md | ||
| appveyor-install-dependencies.ps1 | ||
| change-scala-version.sh | ||
| check-license | ||
| checkstyle-suppressions.xml | ||
| checkstyle.xml | ||
| github_jira_sync.py | ||
| lint-java | ||
| lint-python | ||
| lint-r | ||
| lint-r.R | ||
| lint-scala | ||
| make-distribution.sh | ||
| merge_spark_pr.py | ||
| mima | ||
| pip-sanity-check.py | ||
| README.md | ||
| requirements.txt | ||
| run-pip-tests | ||
| run-tests | ||
| run-tests-jenkins | ||
| run-tests-jenkins.py | ||
| run-tests.py | ||
| sbt-checkstyle | ||
| scalafmt | ||
| scalastyle | ||
| test-dependencies.sh | ||
| tox.ini | ||
Spark Developer Scripts
This directory contains scripts useful to developers when packaging, testing, or committing to Spark.
Many of these scripts require Apache credentials to work correctly.