[SPARK-36026][BUILD][K8S] Upgrade kubernetes-client to 5.5.0
### What changes were proposed in this pull request? Upgrading the kubernetes-client to 5.5.0 ### Why are the changes needed? There are [several bugfixes](https://github.com/fabric8io/kubernetes-client/releases/tag/v5.5.0) but the main reason is version 5.5.0 contains [Support HTTP operation retry with exponential backoff (for status code >= 500)](https://github.com/fabric8io/kubernetes-client/issues/3087). ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? By running the integration tests including `persistentVolume` tests: ``` ./resource-managers/kubernetes/integration-tests/dev/dev-run-integration-tests.sh \ --spark-tgz $TARBALL_TO_TEST --hadoop-profile $HADOOP_PROFILE --exclude-tags r --include-tags persistentVolume ... [INFO] --- scalatest-maven-plugin:2.0.0:test (integration-test) spark-kubernetes-integration-tests_2.12 --- Discovery starting. Discovery completed in 413 milliseconds. Run starting. Expected test count is: 26 KubernetesSuite: - Run SparkPi with no resources - Run SparkPi with a very long application name. - Use SparkLauncher.NO_RESOURCE - Run SparkPi with a master URL without a scheme. - Run SparkPi with an argument. - Run SparkPi with custom labels, annotations, and environment variables. - All pods have the same service account by default - Run extraJVMOptions check on driver - Run SparkRemoteFileTest using a remote data file - Verify logging configuration is picked from the provided SPARK_CONF_DIR/log4j.properties - Run SparkPi with env and mount secrets. - Run PySpark on simple pi.py example - Run PySpark to test a pyfiles example - Run PySpark with memory customization - Run in client mode. - Start pod creation from template - PVs with local storage - Launcher client dependencies - SPARK-33615: Launcher client archives - SPARK-33748: Launcher python client respecting PYSPARK_PYTHON - SPARK-33748: Launcher python client respecting spark.pyspark.python and spark.pyspark.driver.python - Launcher python client dependencies using a zip file - Test basic decommissioning - Test basic decommissioning with shuffle cleanup - Test decommissioning with dynamic allocation & shuffle cleanups - Test decommissioning timeouts Run completed in 18 minutes, 34 seconds. Total number of tests run: 26 Suites: completed 2, aborted 0 Tests: succeeded 26, failed 0, canceled 0, ignored 0, pending 0 All tests passed. ``` Checked the compatibility matrix and the same k8s versions are supported as were by version 5.4.1. Closes #33233 from attilapiros/SPARK-36026. Authored-by: attilapiros <piros.attila.zsolt@gmail.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
This commit is contained in:
attilapiros
Hyukjin Kwon
parent
7566db6033
commit
bad6f89ae2
|
|
@ -154,27 +154,27 @@ jsr305/3.0.0//jsr305-3.0.0.jar
|
|||
jta/1.1//jta-1.1.jar
|
||||
jul-to-slf4j/1.7.30//jul-to-slf4j-1.7.30.jar
|
||||
kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar
|
||||
kubernetes-client/5.4.1//kubernetes-client-5.4.1.jar
|
||||
kubernetes-model-admissionregistration/5.4.1//kubernetes-model-admissionregistration-5.4.1.jar
|
||||
kubernetes-model-apiextensions/5.4.1//kubernetes-model-apiextensions-5.4.1.jar
|
||||
kubernetes-model-apps/5.4.1//kubernetes-model-apps-5.4.1.jar
|
||||
kubernetes-model-autoscaling/5.4.1//kubernetes-model-autoscaling-5.4.1.jar
|
||||
kubernetes-model-batch/5.4.1//kubernetes-model-batch-5.4.1.jar
|
||||
kubernetes-model-certificates/5.4.1//kubernetes-model-certificates-5.4.1.jar
|
||||
kubernetes-model-common/5.4.1//kubernetes-model-common-5.4.1.jar
|
||||
kubernetes-model-coordination/5.4.1//kubernetes-model-coordination-5.4.1.jar
|
||||
kubernetes-model-core/5.4.1//kubernetes-model-core-5.4.1.jar
|
||||
kubernetes-model-discovery/5.4.1//kubernetes-model-discovery-5.4.1.jar
|
||||
kubernetes-model-events/5.4.1//kubernetes-model-events-5.4.1.jar
|
||||
kubernetes-model-extensions/5.4.1//kubernetes-model-extensions-5.4.1.jar
|
||||
kubernetes-model-flowcontrol/5.4.1//kubernetes-model-flowcontrol-5.4.1.jar
|
||||
kubernetes-model-metrics/5.4.1//kubernetes-model-metrics-5.4.1.jar
|
||||
kubernetes-model-networking/5.4.1//kubernetes-model-networking-5.4.1.jar
|
||||
kubernetes-model-node/5.4.1//kubernetes-model-node-5.4.1.jar
|
||||
kubernetes-model-policy/5.4.1//kubernetes-model-policy-5.4.1.jar
|
||||
kubernetes-model-rbac/5.4.1//kubernetes-model-rbac-5.4.1.jar
|
||||
kubernetes-model-scheduling/5.4.1//kubernetes-model-scheduling-5.4.1.jar
|
||||
kubernetes-model-storageclass/5.4.1//kubernetes-model-storageclass-5.4.1.jar
|
||||
kubernetes-client/5.5.0//kubernetes-client-5.5.0.jar
|
||||
kubernetes-model-admissionregistration/5.5.0//kubernetes-model-admissionregistration-5.5.0.jar
|
||||
kubernetes-model-apiextensions/5.5.0//kubernetes-model-apiextensions-5.5.0.jar
|
||||
kubernetes-model-apps/5.5.0//kubernetes-model-apps-5.5.0.jar
|
||||
kubernetes-model-autoscaling/5.5.0//kubernetes-model-autoscaling-5.5.0.jar
|
||||
kubernetes-model-batch/5.5.0//kubernetes-model-batch-5.5.0.jar
|
||||
kubernetes-model-certificates/5.5.0//kubernetes-model-certificates-5.5.0.jar
|
||||
kubernetes-model-common/5.5.0//kubernetes-model-common-5.5.0.jar
|
||||
kubernetes-model-coordination/5.5.0//kubernetes-model-coordination-5.5.0.jar
|
||||
kubernetes-model-core/5.5.0//kubernetes-model-core-5.5.0.jar
|
||||
kubernetes-model-discovery/5.5.0//kubernetes-model-discovery-5.5.0.jar
|
||||
kubernetes-model-events/5.5.0//kubernetes-model-events-5.5.0.jar
|
||||
kubernetes-model-extensions/5.5.0//kubernetes-model-extensions-5.5.0.jar
|
||||
kubernetes-model-flowcontrol/5.5.0//kubernetes-model-flowcontrol-5.5.0.jar
|
||||
kubernetes-model-metrics/5.5.0//kubernetes-model-metrics-5.5.0.jar
|
||||
kubernetes-model-networking/5.5.0//kubernetes-model-networking-5.5.0.jar
|
||||
kubernetes-model-node/5.5.0//kubernetes-model-node-5.5.0.jar
|
||||
kubernetes-model-policy/5.5.0//kubernetes-model-policy-5.5.0.jar
|
||||
kubernetes-model-rbac/5.5.0//kubernetes-model-rbac-5.5.0.jar
|
||||
kubernetes-model-scheduling/5.5.0//kubernetes-model-scheduling-5.5.0.jar
|
||||
kubernetes-model-storageclass/5.5.0//kubernetes-model-storageclass-5.5.0.jar
|
||||
lapack/2.2.0//lapack-2.2.0.jar
|
||||
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
|
||||
libfb303/0.9.3//libfb303-0.9.3.jar
|
||||
|
|
|
|||
|
|
@ -125,27 +125,27 @@ jsr305/3.0.0//jsr305-3.0.0.jar
|
|||
jta/1.1//jta-1.1.jar
|
||||
jul-to-slf4j/1.7.30//jul-to-slf4j-1.7.30.jar
|
||||
kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar
|
||||
kubernetes-client/5.4.1//kubernetes-client-5.4.1.jar
|
||||
kubernetes-model-admissionregistration/5.4.1//kubernetes-model-admissionregistration-5.4.1.jar
|
||||
kubernetes-model-apiextensions/5.4.1//kubernetes-model-apiextensions-5.4.1.jar
|
||||
kubernetes-model-apps/5.4.1//kubernetes-model-apps-5.4.1.jar
|
||||
kubernetes-model-autoscaling/5.4.1//kubernetes-model-autoscaling-5.4.1.jar
|
||||
kubernetes-model-batch/5.4.1//kubernetes-model-batch-5.4.1.jar
|
||||
kubernetes-model-certificates/5.4.1//kubernetes-model-certificates-5.4.1.jar
|
||||
kubernetes-model-common/5.4.1//kubernetes-model-common-5.4.1.jar
|
||||
kubernetes-model-coordination/5.4.1//kubernetes-model-coordination-5.4.1.jar
|
||||
kubernetes-model-core/5.4.1//kubernetes-model-core-5.4.1.jar
|
||||
kubernetes-model-discovery/5.4.1//kubernetes-model-discovery-5.4.1.jar
|
||||
kubernetes-model-events/5.4.1//kubernetes-model-events-5.4.1.jar
|
||||
kubernetes-model-extensions/5.4.1//kubernetes-model-extensions-5.4.1.jar
|
||||
kubernetes-model-flowcontrol/5.4.1//kubernetes-model-flowcontrol-5.4.1.jar
|
||||
kubernetes-model-metrics/5.4.1//kubernetes-model-metrics-5.4.1.jar
|
||||
kubernetes-model-networking/5.4.1//kubernetes-model-networking-5.4.1.jar
|
||||
kubernetes-model-node/5.4.1//kubernetes-model-node-5.4.1.jar
|
||||
kubernetes-model-policy/5.4.1//kubernetes-model-policy-5.4.1.jar
|
||||
kubernetes-model-rbac/5.4.1//kubernetes-model-rbac-5.4.1.jar
|
||||
kubernetes-model-scheduling/5.4.1//kubernetes-model-scheduling-5.4.1.jar
|
||||
kubernetes-model-storageclass/5.4.1//kubernetes-model-storageclass-5.4.1.jar
|
||||
kubernetes-client/5.5.0//kubernetes-client-5.5.0.jar
|
||||
kubernetes-model-admissionregistration/5.5.0//kubernetes-model-admissionregistration-5.5.0.jar
|
||||
kubernetes-model-apiextensions/5.5.0//kubernetes-model-apiextensions-5.5.0.jar
|
||||
kubernetes-model-apps/5.5.0//kubernetes-model-apps-5.5.0.jar
|
||||
kubernetes-model-autoscaling/5.5.0//kubernetes-model-autoscaling-5.5.0.jar
|
||||
kubernetes-model-batch/5.5.0//kubernetes-model-batch-5.5.0.jar
|
||||
kubernetes-model-certificates/5.5.0//kubernetes-model-certificates-5.5.0.jar
|
||||
kubernetes-model-common/5.5.0//kubernetes-model-common-5.5.0.jar
|
||||
kubernetes-model-coordination/5.5.0//kubernetes-model-coordination-5.5.0.jar
|
||||
kubernetes-model-core/5.5.0//kubernetes-model-core-5.5.0.jar
|
||||
kubernetes-model-discovery/5.5.0//kubernetes-model-discovery-5.5.0.jar
|
||||
kubernetes-model-events/5.5.0//kubernetes-model-events-5.5.0.jar
|
||||
kubernetes-model-extensions/5.5.0//kubernetes-model-extensions-5.5.0.jar
|
||||
kubernetes-model-flowcontrol/5.5.0//kubernetes-model-flowcontrol-5.5.0.jar
|
||||
kubernetes-model-metrics/5.5.0//kubernetes-model-metrics-5.5.0.jar
|
||||
kubernetes-model-networking/5.5.0//kubernetes-model-networking-5.5.0.jar
|
||||
kubernetes-model-node/5.5.0//kubernetes-model-node-5.5.0.jar
|
||||
kubernetes-model-policy/5.5.0//kubernetes-model-policy-5.5.0.jar
|
||||
kubernetes-model-rbac/5.5.0//kubernetes-model-rbac-5.5.0.jar
|
||||
kubernetes-model-scheduling/5.5.0//kubernetes-model-scheduling-5.5.0.jar
|
||||
kubernetes-model-storageclass/5.5.0//kubernetes-model-storageclass-5.5.0.jar
|
||||
lapack/2.2.0//lapack-2.2.0.jar
|
||||
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
|
||||
libfb303/0.9.3//libfb303-0.9.3.jar
|
||||
|
|
|
|||
2
pom.xml
2
pom.xml
|
|
@ -203,7 +203,7 @@
|
|||
<arrow.version>2.0.0</arrow.version>
|
||||
<!-- org.fusesource.leveldbjni will be used except on arm64 platform. -->
|
||||
<leveldbjni.group>org.fusesource.leveldbjni</leveldbjni.group>
|
||||
<kubernetes-client.version>5.4.1</kubernetes-client.version>
|
||||
<kubernetes-client.version>5.5.0</kubernetes-client.version>
|
||||
|
||||
<test.java.home>${java.home}</test.java.home>
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue