ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-30715][K8S][TESTS][FOLLOWUP] Update k8s client version in IT as well

Browse source 
### What changes were proposed in this pull request?
This is a follow up for SPARK-30715 . Kubernetes client version in sync in integration-tests and kubernetes/core

### Why are the changes needed?
More than once, the kubernetes client version has gone out of sync between integration tests and kubernetes/core. So brought them up in sync again and added a comment to save us from future need of this additional followup.

### Does this PR introduce any user-facing change?
No

### How was this patch tested?
Manually.

Closes #27948 from ScrapCodes/follow-up-spark-30715.

Authored-by: Prashant Sharma <prashsh1@in.ibm.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This commit is contained in:
Prashant Sharma 2020-03-21 18:26:53 -07:00 committed by Dongjoon Hyun
parent 3a48ea1fe0
commit 3799d2b9d8
No known key found for this signature in database
GPG key ID: EDA00CE834F0FC5C
3 changed files with 6 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
resource-managers/kubernetes/core/pom.xml
View file

@ -29,6 +29,7 @@
<name>Spark Project Kubernetes</name>
<properties>
<sbt.project.name>kubernetes</sbt.project.name>
<!-- Note: Please update the kubernetes client version in kubernetes/integration-tests/pom.xml -->
<kubernetes.client.version>4.7.1</kubernetes.client.version>
</properties>

2
resource-managers/kubernetes/integration-tests/pom.xml
View file

@ -29,7 +29,7 @@
<download-maven-plugin.version>1.3.0</download-maven-plugin.version>
<exec-maven-plugin.version>1.4.0</exec-maven-plugin.version>
<extraScalaTestArgs></extraScalaTestArgs>
<kubernetes-client.version>4.6.4</kubernetes-client.version>
<kubernetes-client.version>4.7.1</kubernetes-client.version>
<scala-maven-plugin.version>3.2.2</scala-maven-plugin.version>
<scalatest-maven-plugin.version>1.0</scalatest-maven-plugin.version>
<sbt.project.name>kubernetes-integration-tests</sbt.project.name>

8
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala
View file

@ -61,15 +61,15 @@ class KubernetesSuite extends SparkFunSuite
protected var appLocator: String = _
// Default memory limit is 1024M + 384M (minimum overhead constant)
private val baseMemory = s"${1024 + 384}Mi"
private val baseMemory = s"${1024 + 384}"
protected val memOverheadConstant = 0.8
private val standardNonJVMMemory = s"${(1024 + 0.4*1024).toInt}Mi"
private val standardNonJVMMemory = s"${(1024 + 0.4*1024).toInt}"
protected val additionalMemory = 200
// 209715200 is 200Mi
protected val additionalMemoryInBytes = 209715200
private val extraDriverTotalMemory = s"${(1024 + memOverheadConstant*1024).toInt}Mi"
private val extraDriverTotalMemory = s"${(1024 + memOverheadConstant*1024).toInt}"
private val extraExecTotalMemory =
s"${(1024 + memOverheadConstant*1024 + additionalMemory).toInt}Mi"
s"${(1024 + memOverheadConstant*1024 + additionalMemory).toInt}"
/**
* Build the image ref for the given image name, taking the repo and tag from the