[SPARK-33237][K8S][TESTS] Use default Hadoop-3.2 profile from K8s IT Jenkins job
### What changes were proposed in this pull request? This PR aims to use `hadoop-3.2` profile in K8s IT Jenkins jobs. - [x] Switch the default value of `HADOOP_PROFILE` from `hadoop-2.7` to `hadoop-3.2`. - [x] Remove `-Phadoop2.7` from Jenkins K8s IT job. - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/configure **BEFORE** ``` ./dev/make-distribution.sh --name ${DATE}-${REVISION} --r --pip --tgz -DzincPort=${ZINC_PORT} \ -Phadoop-2.7 -Pkubernetes -Pkinesis-asl -Phive -Phive-thriftserver ``` **AFTER** ``` ./dev/make-distribution.sh --name ${DATE}-${REVISION} --r --pip --tgz -DzincPort=${ZINC_PORT} \ -Pkubernetes -Pkinesis-asl -Phive -Phive-thriftserver ``` ### Why are the changes needed? Since Apache Spark 3.1.0, Hadoop 3 is the default. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Check the Jenkins K8s IT log and result. - https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34899/ ``` + /home/jenkins/workspace/SparkPullRequestBuilder-K8s/build/mvn clean package -DskipTests -DzincPort=4021 -Pkubernetes -Pkinesis-asl -Phive -Phive-thriftserver Using `mvn` from path: /home/jenkins/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.6.3/bin/mvn [INFO] Scanning for projects... [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] ``` Closes #30153 from dongjoon-hyun/SPARK-33237. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
This commit is contained in:
parent
02fa19f102
commit
afa6aee4f5
|
@ -35,7 +35,7 @@ CONTEXT=
|
|||
INCLUDE_TAGS="k8s"
|
||||
EXCLUDE_TAGS=
|
||||
JAVA_VERSION="8"
|
||||
HADOOP_PROFILE="hadoop-2.7"
|
||||
HADOOP_PROFILE="hadoop-3.2"
|
||||
MVN="$TEST_ROOT_DIR/build/mvn"
|
||||
|
||||
SCALA_VERSION=$("$MVN" help:evaluate -Dexpression=scala.binary.version 2>/dev/null\
|
||||
|
|
Loading…
Reference in a new issue