e1ee3fb72f
### What changes were proposed in this pull request? - Reverts commit1f94bf4
andd6be46e
- Switches python to python3 in Docker release image. ### Why are the changes needed? `dev/make-distribution.sh` and `python/setup.py` are use python3. https://github.com/apache/spark/pull/26844/files#diff-ba2c046d92a1d2b5b417788bfb5cb5f8L236 https://github.com/apache/spark/pull/26330/files#diff-8cf6167d58ce775a08acafcfe6f40966 ### Does this PR introduce any user-facing change? No. ### How was this patch tested? manual test: ``` yumwangubuntu-3513086:~/spark$ dev/create-release/do-release-docker.sh -n -d /home/yumwang/spark-release Output directory already exists. Overwrite and continue? [y/n] y Branch [branch-2.4]: master Current branch version is 3.0.0-SNAPSHOT. Release [3.0.0]: 3.0.0-preview2 RC # [1]: This is a dry run. Please confirm the ref that will be built for testing. Ref [master]: ASF user [yumwang]: Full name [Yuming Wang]: GPG key [yumwangapache.org]: DBD447010C1B4F7DAD3F7DFD6E1B4122F6A3A338 ================ Release details: BRANCH: master VERSION: 3.0.0-preview2 TAG: v3.0.0-preview2-rc1 NEXT: 3.0.1-SNAPSHOT ASF USER: yumwang GPG KEY: DBD447010C1B4F7DAD3F7DFD6E1B4122F6A3A338 FULL NAME: Yuming Wang E-MAIL: yumwangapache.org ================ Is this info correct [y/n]? y GPG passphrase: ======================== = Building spark-rm image with tag latest... Command: docker build -t spark-rm:latest --build-arg UID=110302528 /home/yumwang/spark/dev/create-release/spark-rm Log file: docker-build.log Building v3.0.0-preview2-rc1; output will be at /home/yumwang/spark-release/output gpg: directory '/home/spark-rm/.gnupg' created gpg: keybox '/home/spark-rm/.gnupg/pubring.kbx' created gpg: /home/spark-rm/.gnupg/trustdb.gpg: trustdb created gpg: key 6E1B4122F6A3A338: public key "Yuming Wang <yumwangapache.org>" imported gpg: key 6E1B4122F6A3A338: secret key imported gpg: Total number processed: 1 gpg: imported: 1 gpg: secret keys read: 1 gpg: secret keys imported: 1 ======================== = Creating release tag v3.0.0-preview2-rc1... Command: /opt/spark-rm/release-tag.sh Log file: tag.log It may take some time for the tag to be synchronized to github. Press enter when you've verified that the new tag (v3.0.0-preview2-rc1) is available. ======================== = Building Spark... Command: /opt/spark-rm/release-build.sh package Log file: build.log ======================== = Building documentation... Command: /opt/spark-rm/release-build.sh docs Log file: docs.log ======================== = Publishing release Command: /opt/spark-rm/release-build.sh publish-release Log file: publish.log ``` Generated doc: ![image](https://user-images.githubusercontent.com/5399861/70693075-a7723100-1cf7-11ea-9f88-9356a02349a1.png) Closes #26848 from wangyum/SPARK-30216. Authored-by: Yuming Wang <yumwang@ebay.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
180 lines
6.7 KiB
Makefile
180 lines
6.7 KiB
Makefile
# Makefile for Sphinx documentation
|
|
#
|
|
|
|
# You can set these variables from the command line.
|
|
SPHINXOPTS ?=
|
|
SPHINXBUILD ?= sphinx-build
|
|
PAPER ?=
|
|
BUILDDIR ?= _build
|
|
|
|
export PYTHONPATH=$(realpath ..):$(realpath ../lib/py4j-0.10.8.1-src.zip)
|
|
|
|
# User-friendly check for sphinx-build
|
|
ifeq ($(shell which $(SPHINXBUILD) >/dev/null 2>&1; echo $$?), 1)
|
|
$(error The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed, then set the SPHINXBUILD environment variable to point to the full path of the '$(SPHINXBUILD)' executable. Alternatively you can add the directory with the executable to your PATH. If you don't have Sphinx installed, grab it from http://sphinx-doc.org/)
|
|
endif
|
|
|
|
# Internal variables.
|
|
PAPEROPT_a4 = -D latex_paper_size=a4
|
|
PAPEROPT_letter = -D latex_paper_size=letter
|
|
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
|
# the i18n builder cannot share the environment and doctrees with the others
|
|
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
|
|
|
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
|
|
|
|
help:
|
|
@echo "Please use \`make <target>' where <target> is one of"
|
|
@echo " html to make standalone HTML files"
|
|
@echo " dirhtml to make HTML files named index.html in directories"
|
|
@echo " singlehtml to make a single large HTML file"
|
|
@echo " pickle to make pickle files"
|
|
@echo " json to make JSON files"
|
|
@echo " htmlhelp to make HTML files and a HTML help project"
|
|
@echo " qthelp to make HTML files and a qthelp project"
|
|
@echo " devhelp to make HTML files and a Devhelp project"
|
|
@echo " epub to make an epub"
|
|
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
|
|
@echo " latexpdf to make LaTeX files and run them through pdflatex"
|
|
@echo " latexpdfja to make LaTeX files and run them through platex/dvipdfmx"
|
|
@echo " text to make text files"
|
|
@echo " man to make manual pages"
|
|
@echo " texinfo to make Texinfo files"
|
|
@echo " info to make Texinfo files and run them through makeinfo"
|
|
@echo " gettext to make PO message catalogs"
|
|
@echo " changes to make an overview of all changed/added/deprecated items"
|
|
@echo " xml to make Docutils-native XML files"
|
|
@echo " pseudoxml to make pseudoxml-XML files for display purposes"
|
|
@echo " linkcheck to check all external links for integrity"
|
|
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
|
|
|
|
clean:
|
|
rm -rf $(BUILDDIR)/*
|
|
|
|
html:
|
|
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
|
|
@echo
|
|
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
|
|
|
|
dirhtml:
|
|
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
|
|
@echo
|
|
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
|
|
|
|
singlehtml:
|
|
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
|
|
@echo
|
|
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
|
|
|
|
pickle:
|
|
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
|
|
@echo
|
|
@echo "Build finished; now you can process the pickle files."
|
|
|
|
json:
|
|
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
|
|
@echo
|
|
@echo "Build finished; now you can process the JSON files."
|
|
|
|
htmlhelp:
|
|
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
|
|
@echo
|
|
@echo "Build finished; now you can run HTML Help Workshop with the" \
|
|
".hhp project file in $(BUILDDIR)/htmlhelp."
|
|
|
|
qthelp:
|
|
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
|
|
@echo
|
|
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
|
|
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
|
|
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/pyspark.qhcp"
|
|
@echo "To view the help file:"
|
|
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/pyspark.qhc"
|
|
|
|
devhelp:
|
|
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
|
|
@echo
|
|
@echo "Build finished."
|
|
@echo "To view the help file:"
|
|
@echo "# mkdir -p $$HOME/.local/share/devhelp/pyspark"
|
|
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/pyspark"
|
|
@echo "# devhelp"
|
|
|
|
epub:
|
|
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
|
|
@echo
|
|
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
|
|
|
|
latex:
|
|
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
|
|
@echo
|
|
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
|
|
@echo "Run \`make' in that directory to run these through (pdf)latex" \
|
|
"(use \`make latexpdf' here to do that automatically)."
|
|
|
|
latexpdf:
|
|
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
|
|
@echo "Running LaTeX files through pdflatex..."
|
|
$(MAKE) -C $(BUILDDIR)/latex all-pdf
|
|
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
|
|
|
|
latexpdfja:
|
|
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
|
|
@echo "Running LaTeX files through platex and dvipdfmx..."
|
|
$(MAKE) -C $(BUILDDIR)/latex all-pdf-ja
|
|
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
|
|
|
|
text:
|
|
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
|
|
@echo
|
|
@echo "Build finished. The text files are in $(BUILDDIR)/text."
|
|
|
|
man:
|
|
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
|
|
@echo
|
|
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
|
|
|
|
texinfo:
|
|
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
|
|
@echo
|
|
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
|
|
@echo "Run \`make' in that directory to run these through makeinfo" \
|
|
"(use \`make info' here to do that automatically)."
|
|
|
|
info:
|
|
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
|
|
@echo "Running Texinfo files through makeinfo..."
|
|
make -C $(BUILDDIR)/texinfo info
|
|
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
|
|
|
|
gettext:
|
|
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
|
|
@echo
|
|
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
|
|
|
|
changes:
|
|
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
|
|
@echo
|
|
@echo "The overview file is in $(BUILDDIR)/changes."
|
|
|
|
linkcheck:
|
|
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
|
|
@echo
|
|
@echo "Link check complete; look for any errors in the above output " \
|
|
"or in $(BUILDDIR)/linkcheck/output.txt."
|
|
|
|
doctest:
|
|
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
|
|
@echo "Testing of doctests in the sources finished, look at the " \
|
|
"results in $(BUILDDIR)/doctest/output.txt."
|
|
|
|
xml:
|
|
$(SPHINXBUILD) -b xml $(ALLSPHINXOPTS) $(BUILDDIR)/xml
|
|
@echo
|
|
@echo "Build finished. The XML files are in $(BUILDDIR)/xml."
|
|
|
|
pseudoxml:
|
|
$(SPHINXBUILD) -b pseudoxml $(ALLSPHINXOPTS) $(BUILDDIR)/pseudoxml
|
|
@echo
|
|
@echo "Build finished. The pseudo-XML files are in $(BUILDDIR)/pseudoxml."
|