ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-31766][K8S][TESTS] Add Spark version prefix to K8s UUID test image tag

Browse source 
### What changes were proposed in this pull request?

This PR aims to add Spark version prefix during generating test image tag for K8s integration testing.

### Why are the changes needed?

This helps to distinguish the images by version.

**BEFORE**
```
$ docker images | grep kubespark
kubespark/spark-py  F7188CBD-AE08-4705-9C8A-D0DD3DC8B86F  ...
kubespark/spark     F7188CBD-AE08-4705-9C8A-D0DD3DC8B86F  ...
```

**AFTER**
```
$ docker images | grep kubespark
kubespark/spark-py  3.1.0-SNAPSHOT_F7188CBD-AE08-4705-9C8A-D0DD3DC8B86F ...
kubespark/spark     3.1.0-SNAPSHOT_F7188CBD-AE08-4705-9C8A-D0DD3DC8B86F ...
```

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the K8s integration test.

```
...
Successfully tagged kubespark/spark:3.1.0-SNAPSHOT_688b46c8-c119-404d-aadb-d05a14262db7
...
Successfully tagged kubespark/spark-py:3.1.0-SNAPSHOT_688b46c8-c119-404d-aadb-d05a14262db7
...
Successfully tagged kubespark/spark-r:3.1.0-SNAPSHOT_688b46c8-c119-404d-aadb-d05a14262db7
```

Closes #28587 from dongjoon-hyun/SPARK-31766.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
Dongjoon Hyun 2020-05-20 14:55:23 +09:00 committed by HyukjinKwon
parent dc3a606fbd
commit b7947e0285
1 changed files with 6 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

7
resource-managers/kubernetes/integration-tests/scripts/setup-integration-test-env.sh
View file

@ -25,6 +25,7 @@ IMAGE_REPO="docker.io/kubespark"
IMAGE_TAG="N/A"
JAVA_IMAGE_TAG="8-jre-slim"
SPARK_TGZ="N/A"
MVN="$TEST_ROOT_DIR/build/mvn"
# Parse arguments
while (( "$#" )); do
@ -84,7 +85,11 @@ fi
# If there is a specific Spark image skip building and extraction/copy
if [[ $IMAGE_TAG == "N/A" ]];
then
IMAGE_TAG=$(uuidgen);
VERSION=$("$MVN" help:evaluate -Dexpression=project.version \
| grep -v "INFO"\
| grep -v "WARNING"\
| tail -n 1)
IMAGE_TAG=${VERSION}_$(uuidgen)
cd $SPARK_INPUT_DIR
# OpenJDK base-image tag (e.g. 8-jre-slim, 11-jre-slim)