From 29e09a83b7c6babdd2cc5688ce27982de4c3cd46 Mon Sep 17 00:00:00 2001 From: Nicholas Chammas Date: Wed, 4 Dec 2019 17:31:23 -0600 Subject: [PATCH] [SPARK-30084][DOCS] Document how to trigger Jekyll build on Python API doc changes ### What changes were proposed in this pull request? This PR adds a note to the docs README showing how to get Jekyll to automatically pick up changes to the Python API docs. ### Why are the changes needed? `jekyll serve --watch` doesn't watch for changes to the API docs. Without the technique documented in this note, or something equivalent, developers have to manually retrigger a Jekyll build any time they update the Python API docs. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? I tested this PR manually by making changes to Python docstrings and confirming that Jekyll automatically picks them up and serves them locally. Closes #26719 from nchammas/SPARK-30084-watch-api-docs. Authored-by: Nicholas Chammas Signed-off-by: Sean Owen --- .gitignore | 3 ++- docs/README.md | 14 ++++++++++++++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/.gitignore b/.gitignore index ae20c85ebe..798e8acc4d 100644 --- a/.gitignore +++ b/.gitignore @@ -45,7 +45,7 @@ dev/create-release/*final dev/create-release/*txt dev/pr-deps/ dist/ -docs/_site +docs/_site/ docs/api sql/docs sql/site @@ -63,6 +63,7 @@ project/plugins/target/ python/lib/pyspark.zip python/.eggs/ python/deps +python/docs/_site/ python/test_coverage/coverage_data python/test_coverage/htmlcov python/pyspark/python diff --git a/docs/README.md b/docs/README.md index 0bb1ada716..ef849d53da 100644 --- a/docs/README.md +++ b/docs/README.md @@ -102,3 +102,17 @@ using [MkDocs](https://www.mkdocs.org/). NOTE: To skip the step of building and copying over the Scala, Java, Python, R and SQL API docs, run `SKIP_API=1 jekyll build`. In addition, `SKIP_SCALADOC=1`, `SKIP_PYTHONDOC=1`, `SKIP_RDOC=1` and `SKIP_SQLDOC=1` can be used to skip a single step of the corresponding language. `SKIP_SCALADOC` indicates skipping both the Scala and Java docs. + +### Automatically Rebuilding API Docs + +`jekyll serve --watch` will only watch what's in `docs/`, and it won't follow symlinks. That means it won't monitor your API docs under `python/docs` or elsewhere. + +To work around this limitation for Python, install [`entr`](http://eradman.com/entrproject/) and run the following in a separate shell: + +```sh +cd "$SPARK_HOME/python/docs" +find .. -type f -name '*.py' \ +| entr -s 'make html && cp -r _build/html/. ../../docs/api/python' +``` + +Whenever there is a change to your Python code, `entr` will automatically rebuild the Python API docs and copy them to `docs/`, thus triggering a Jekyll update.