[SPARK-34486][K8S] Upgrade kubernetes-client to 4.13.2
### What changes were proposed in this pull request? This PR aims to upgrade `kubernetes-client` library from 4.12.0 to 4.13.2 for Apache Spark 3.2.0. ### Why are the changes needed? This will bring [K8s 1.19.1](https://github.com/fabric8io/kubernetes-client/pull/2541) models officially and the latest bug fixes. - https://github.com/fabric8io/kubernetes-client/releases/tag/v4.13.0 - https://github.com/fabric8io/kubernetes-client/releases/tag/v4.13.1 - https://github.com/fabric8io/kubernetes-client/releases/tag/v4.13.2 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass the K8s IT and UT. ``` KubernetesSuite: - Run SparkPi with no resources - Run SparkPi with a very long application name. - Use SparkLauncher.NO_RESOURCE - Run SparkPi with a master URL without a scheme. - Run SparkPi with an argument. - Run SparkPi with custom labels, annotations, and environment variables. - All pods have the same service account by default - Run extraJVMOptions check on driver - Run SparkRemoteFileTest using a remote data file - Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties - Run SparkPi with env and mount secrets. - Run PySpark on simple pi.py example - Run PySpark to test a pyfiles example - Run PySpark with memory customization - Run in client mode. - Start pod creation from template - PVs with local storage - Launcher client dependencies - SPARK-33615: Launcher client archives - SPARK-33748: Launcher python client respecting PYSPARK_PYTHON - SPARK-33748: Launcher python client respecting spark.pyspark.python and spark.pyspark.driver.python - Launcher python client dependencies using a zip file - Test basic decommissioning - Test basic decommissioning with shuffle cleanup - Test decommissioning with dynamic allocation & shuffle cleanups - Test decommissioning timeouts - Run SparkR on simple dataframe.R example Run completed in 19 minutes, 25 seconds. Total number of tests run: 27 Suites: completed 2, aborted 0 Tests: succeeded 27, failed 0, canceled 0, ignored 0, pending 0 All tests passed. ``` Closes #31602 from dongjoon-hyun/SPARK-34486. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
parent
546d2eb5d4
commit
020e84e92f
|
@ -156,26 +156,27 @@ jsr305/3.0.0//jsr305-3.0.0.jar
|
|||
jta/1.1//jta-1.1.jar
|
||||
jul-to-slf4j/1.7.30//jul-to-slf4j-1.7.30.jar
|
||||
kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar
|
||||
kubernetes-client/4.12.0//kubernetes-client-4.12.0.jar
|
||||
kubernetes-model-admissionregistration/4.12.0//kubernetes-model-admissionregistration-4.12.0.jar
|
||||
kubernetes-model-apiextensions/4.12.0//kubernetes-model-apiextensions-4.12.0.jar
|
||||
kubernetes-model-apps/4.12.0//kubernetes-model-apps-4.12.0.jar
|
||||
kubernetes-model-autoscaling/4.12.0//kubernetes-model-autoscaling-4.12.0.jar
|
||||
kubernetes-model-batch/4.12.0//kubernetes-model-batch-4.12.0.jar
|
||||
kubernetes-model-certificates/4.12.0//kubernetes-model-certificates-4.12.0.jar
|
||||
kubernetes-model-common/4.12.0//kubernetes-model-common-4.12.0.jar
|
||||
kubernetes-model-coordination/4.12.0//kubernetes-model-coordination-4.12.0.jar
|
||||
kubernetes-model-core/4.12.0//kubernetes-model-core-4.12.0.jar
|
||||
kubernetes-model-discovery/4.12.0//kubernetes-model-discovery-4.12.0.jar
|
||||
kubernetes-model-events/4.12.0//kubernetes-model-events-4.12.0.jar
|
||||
kubernetes-model-extensions/4.12.0//kubernetes-model-extensions-4.12.0.jar
|
||||
kubernetes-model-metrics/4.12.0//kubernetes-model-metrics-4.12.0.jar
|
||||
kubernetes-model-networking/4.12.0//kubernetes-model-networking-4.12.0.jar
|
||||
kubernetes-model-policy/4.12.0//kubernetes-model-policy-4.12.0.jar
|
||||
kubernetes-model-rbac/4.12.0//kubernetes-model-rbac-4.12.0.jar
|
||||
kubernetes-model-scheduling/4.12.0//kubernetes-model-scheduling-4.12.0.jar
|
||||
kubernetes-model-settings/4.12.0//kubernetes-model-settings-4.12.0.jar
|
||||
kubernetes-model-storageclass/4.12.0//kubernetes-model-storageclass-4.12.0.jar
|
||||
kubernetes-client/4.13.2//kubernetes-client-4.13.2.jar
|
||||
kubernetes-model-admissionregistration/4.13.2//kubernetes-model-admissionregistration-4.13.2.jar
|
||||
kubernetes-model-apiextensions/4.13.2//kubernetes-model-apiextensions-4.13.2.jar
|
||||
kubernetes-model-apps/4.13.2//kubernetes-model-apps-4.13.2.jar
|
||||
kubernetes-model-autoscaling/4.13.2//kubernetes-model-autoscaling-4.13.2.jar
|
||||
kubernetes-model-batch/4.13.2//kubernetes-model-batch-4.13.2.jar
|
||||
kubernetes-model-certificates/4.13.2//kubernetes-model-certificates-4.13.2.jar
|
||||
kubernetes-model-common/4.13.2//kubernetes-model-common-4.13.2.jar
|
||||
kubernetes-model-coordination/4.13.2//kubernetes-model-coordination-4.13.2.jar
|
||||
kubernetes-model-core/4.13.2//kubernetes-model-core-4.13.2.jar
|
||||
kubernetes-model-discovery/4.13.2//kubernetes-model-discovery-4.13.2.jar
|
||||
kubernetes-model-events/4.13.2//kubernetes-model-events-4.13.2.jar
|
||||
kubernetes-model-extensions/4.13.2//kubernetes-model-extensions-4.13.2.jar
|
||||
kubernetes-model-metrics/4.13.2//kubernetes-model-metrics-4.13.2.jar
|
||||
kubernetes-model-networking/4.13.2//kubernetes-model-networking-4.13.2.jar
|
||||
kubernetes-model-node/4.13.2//kubernetes-model-node-4.13.2.jar
|
||||
kubernetes-model-policy/4.13.2//kubernetes-model-policy-4.13.2.jar
|
||||
kubernetes-model-rbac/4.13.2//kubernetes-model-rbac-4.13.2.jar
|
||||
kubernetes-model-scheduling/4.13.2//kubernetes-model-scheduling-4.13.2.jar
|
||||
kubernetes-model-settings/4.13.2//kubernetes-model-settings-4.13.2.jar
|
||||
kubernetes-model-storageclass/4.13.2//kubernetes-model-storageclass-4.13.2.jar
|
||||
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
|
||||
libfb303/0.9.3//libfb303-0.9.3.jar
|
||||
libthrift/0.12.0//libthrift-0.12.0.jar
|
||||
|
|
|
@ -126,26 +126,27 @@ jsr305/3.0.0//jsr305-3.0.0.jar
|
|||
jta/1.1//jta-1.1.jar
|
||||
jul-to-slf4j/1.7.30//jul-to-slf4j-1.7.30.jar
|
||||
kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar
|
||||
kubernetes-client/4.12.0//kubernetes-client-4.12.0.jar
|
||||
kubernetes-model-admissionregistration/4.12.0//kubernetes-model-admissionregistration-4.12.0.jar
|
||||
kubernetes-model-apiextensions/4.12.0//kubernetes-model-apiextensions-4.12.0.jar
|
||||
kubernetes-model-apps/4.12.0//kubernetes-model-apps-4.12.0.jar
|
||||
kubernetes-model-autoscaling/4.12.0//kubernetes-model-autoscaling-4.12.0.jar
|
||||
kubernetes-model-batch/4.12.0//kubernetes-model-batch-4.12.0.jar
|
||||
kubernetes-model-certificates/4.12.0//kubernetes-model-certificates-4.12.0.jar
|
||||
kubernetes-model-common/4.12.0//kubernetes-model-common-4.12.0.jar
|
||||
kubernetes-model-coordination/4.12.0//kubernetes-model-coordination-4.12.0.jar
|
||||
kubernetes-model-core/4.12.0//kubernetes-model-core-4.12.0.jar
|
||||
kubernetes-model-discovery/4.12.0//kubernetes-model-discovery-4.12.0.jar
|
||||
kubernetes-model-events/4.12.0//kubernetes-model-events-4.12.0.jar
|
||||
kubernetes-model-extensions/4.12.0//kubernetes-model-extensions-4.12.0.jar
|
||||
kubernetes-model-metrics/4.12.0//kubernetes-model-metrics-4.12.0.jar
|
||||
kubernetes-model-networking/4.12.0//kubernetes-model-networking-4.12.0.jar
|
||||
kubernetes-model-policy/4.12.0//kubernetes-model-policy-4.12.0.jar
|
||||
kubernetes-model-rbac/4.12.0//kubernetes-model-rbac-4.12.0.jar
|
||||
kubernetes-model-scheduling/4.12.0//kubernetes-model-scheduling-4.12.0.jar
|
||||
kubernetes-model-settings/4.12.0//kubernetes-model-settings-4.12.0.jar
|
||||
kubernetes-model-storageclass/4.12.0//kubernetes-model-storageclass-4.12.0.jar
|
||||
kubernetes-client/4.13.2//kubernetes-client-4.13.2.jar
|
||||
kubernetes-model-admissionregistration/4.13.2//kubernetes-model-admissionregistration-4.13.2.jar
|
||||
kubernetes-model-apiextensions/4.13.2//kubernetes-model-apiextensions-4.13.2.jar
|
||||
kubernetes-model-apps/4.13.2//kubernetes-model-apps-4.13.2.jar
|
||||
kubernetes-model-autoscaling/4.13.2//kubernetes-model-autoscaling-4.13.2.jar
|
||||
kubernetes-model-batch/4.13.2//kubernetes-model-batch-4.13.2.jar
|
||||
kubernetes-model-certificates/4.13.2//kubernetes-model-certificates-4.13.2.jar
|
||||
kubernetes-model-common/4.13.2//kubernetes-model-common-4.13.2.jar
|
||||
kubernetes-model-coordination/4.13.2//kubernetes-model-coordination-4.13.2.jar
|
||||
kubernetes-model-core/4.13.2//kubernetes-model-core-4.13.2.jar
|
||||
kubernetes-model-discovery/4.13.2//kubernetes-model-discovery-4.13.2.jar
|
||||
kubernetes-model-events/4.13.2//kubernetes-model-events-4.13.2.jar
|
||||
kubernetes-model-extensions/4.13.2//kubernetes-model-extensions-4.13.2.jar
|
||||
kubernetes-model-metrics/4.13.2//kubernetes-model-metrics-4.13.2.jar
|
||||
kubernetes-model-networking/4.13.2//kubernetes-model-networking-4.13.2.jar
|
||||
kubernetes-model-node/4.13.2//kubernetes-model-node-4.13.2.jar
|
||||
kubernetes-model-policy/4.13.2//kubernetes-model-policy-4.13.2.jar
|
||||
kubernetes-model-rbac/4.13.2//kubernetes-model-rbac-4.13.2.jar
|
||||
kubernetes-model-scheduling/4.13.2//kubernetes-model-scheduling-4.13.2.jar
|
||||
kubernetes-model-settings/4.13.2//kubernetes-model-settings-4.13.2.jar
|
||||
kubernetes-model-storageclass/4.13.2//kubernetes-model-storageclass-4.13.2.jar
|
||||
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
|
||||
libfb303/0.9.3//libfb303-0.9.3.jar
|
||||
libthrift/0.12.0//libthrift-0.12.0.jar
|
||||
|
|
|
@ -30,7 +30,7 @@
|
|||
<properties>
|
||||
<sbt.project.name>kubernetes</sbt.project.name>
|
||||
<!-- Note: Please update the kubernetes client version in kubernetes/integration-tests/pom.xml -->
|
||||
<kubernetes.client.version>4.12.0</kubernetes.client.version>
|
||||
<kubernetes.client.version>4.13.2</kubernetes.client.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
|
|
|
@ -28,7 +28,7 @@
|
|||
<properties>
|
||||
<download-maven-plugin.version>1.3.0</download-maven-plugin.version>
|
||||
<extraScalaTestArgs></extraScalaTestArgs>
|
||||
<kubernetes-client.version>4.12.0</kubernetes-client.version>
|
||||
<kubernetes-client.version>4.13.2</kubernetes-client.version>
|
||||
<sbt.project.name>kubernetes-integration-tests</sbt.project.name>
|
||||
|
||||
<!-- Integration Test Configuration Properties -->
|
||||
|
|
Loading…
Reference in a new issue