ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-34979][PYTHON][DOC] Add PyArrow installation note for PySpark aarch64 user

Browse source 
### What changes were proposed in this pull request?

This patch adds a note for aarch64 user to install the specific pyarrow>=4.0.0.

### Why are the changes needed?

The pyarrow aarch64 support is [introduced](https://github.com/apache/arrow/pull/9285) in [PyArrow 4.0.0](https://github.com/apache/arrow/releases/tag/apache-arrow-4.0.0), and it has been published 27.Apr.2021.

See more in [SPARK-34979](https://issues.apache.org/jira/browse/SPARK-34979).

### Does this PR introduce _any_ user-facing change?
Yes, this doc can help user install arrow on aarch64.

### How was this patch tested?
doc test passed.

Closes #32363 from Yikun/SPARK-34979.

Authored-by: Yikun Jiang <yikunkero@gmail.com>
Signed-off-by: hyukjinkwon <gurwls223@apache.org>
This commit is contained in:
Yikun Jiang 2021-04-28 09:56:17 +09:00 committed by hyukjinkwon
parent 5b77ebb57b
commit 0769049ee1
1 changed files with 8 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

8
python/docs/source/getting_started/install.rst
View file

@ -164,3 +164,11 @@ Package Minimum supported version Note
Note that PySpark requires Java 8 or later with ``JAVA_HOME`` properly set.
If using JDK 11, set ``-Dio.netty.tryReflectionSetAccessible=true`` for Arrow related features and refer
to |downloading|_.
Note for AArch64 (ARM64) users: PyArrow is required by PySpark SQL, but PyArrow support for AArch64
is introduced in PyArrow 4.0.0. If PySpark installation fails on AArch64 due to PyArrow
installation errors, you can install PyArrow >= 4.0.0 as below:
.. code-block:: bash
pip install "pyarrow>=4.0.0" --prefer-binary