spark-instrumented-optimizer/python/docs/source/user_guide
David Li 1237124062 [SPARK-34463][PYSPARK][DOCS] Document caveats of Arrow selfDestruct
### What changes were proposed in this pull request?

As a followup for #29818, document caveats of using the Arrow selfDestruct option in toPandas, which include:
- toPandas() may be slower;
- the resulting dataframe may not support some Pandas operations due to immutable backing arrays.

### Why are the changes needed?

This will hopefully reduce user confusion as with SPARK-34463.

### Does this PR introduce _any_ user-facing change?

Yes - documentation is updated and a config setting description is updated to clearly indicate the config is experimental.

### How was this patch tested?
This is a documentation-only change.

Closes #31738 from lidavidm/spark-34463.

Authored-by: David Li <li.davidm96@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
2021-03-30 13:30:27 +09:00
..
arrow_pandas.rst [SPARK-34463][PYSPARK][DOCS] Document caveats of Arrow selfDestruct 2021-03-30 13:30:27 +09:00
index.rst [SPARK-34818][PYTHON][DOCS] Reorder the items in User Guide at PySpark documentation 2021-03-22 15:53:39 +09:00
python_packaging.rst [SPARK-34190][DOCS] Supplement the description for Python Package Management 2021-01-21 22:15:42 +09:00