ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/resource-managers/kubernetes/integration-tests/tests
History
Liang-Chi Hsieh 37eedf6149 [SPARK-28652][TESTS][K8S] Add python version check for executor 
## What changes were proposed in this pull request?

Current two PySpark version tests in PythonTestsSuite, just test against Python version at driver side. Because the test script doesn't run any spark job requiring python worker, it doesn't actually do version check at worker side. This patch adds pieces of code to the test script, to run a simple job to verify Python version.

## How was this patch tested?

Unit test. Locally manual test.

Closes #25411 from viirya/SPARK-28652.

Lead-authored-by: Liang-Chi Hsieh <viirya@gmail.com>
Co-authored-by: Liang-Chi Hsieh <liangchi@uber.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
2019-08-12 14:43:32 +09:00
..
py_container_checks.py [SPARK-25021][K8S] Add spark.executor.pyspark.memory limit for K8S 2018-09-08 22:18:06 -07:00
pyfiles.py [SPARK-28652][TESTS][K8S] Add python version check for executor 2019-08-12 14:43:32 +09:00
worker_memory_check.py [SPARK-25021][K8S] Add spark.executor.pyspark.memory limit for K8S 2018-09-08 22:18:06 -07:00