678294ddc2
### What changes were proposed in this pull request? This PR proposes to clarify: - `PYSPARK_DRIVER_PYTHON` should not be set for cluster modes in YARN and Kubernates. - `spark.yarn.appMasterEnv.PYSPARK_PYTHON` is not required in YARN. This is just another way to set `PYSPARK_PYTHON` that is specific for a Spark application. ### Why are the changes needed? To clarify what's required and not. ### Does this PR introduce _any_ user-facing change? Yes, this is a user-facing doc change. ### How was this patch tested? Manually tested. Note that this credits to gaborgsomogyi who actually tested and raised a doubt about this offline to me. I also manually tested all again to double check. Closes #30938 from HyukjinKwon/SPARK-33824-followup. Authored-by: Gabor Somogyi <gabor.g.somogyi@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org> |
||
---|---|---|
.. | ||
_static | ||
_templates/autosummary | ||
development | ||
getting_started | ||
migration_guide | ||
reference | ||
user_guide | ||
conf.py | ||
index.rst |