[SPARK-33179][TESTS] Switch default Hadoop profile in run-tests.py
### What changes were proposed in this pull request? This PR aims to switch the default Hadoop profile from `hadoop2.7` to `hadoop3.2` in `dev/run-tests.py` when it's running in local or GitHub Action environments. ### Why are the changes needed? The default Hadoop version is 3.2. We had better be consistent. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually. **BEFORE** ``` % dev/run-tests.py Cannot install SparkR as R was not found in PATH [info] Using build tool sbt with Hadoop profile hadoop2.7 and Hive profile hive2.3 under environment local ``` **AFTER** ``` % dev/run-tests.py Cannot install SparkR as R was not found in PATH [info] Using build tool sbt with Hadoop profile hadoop3.2 and Hive profile hive2.3 under environment local ``` Closes #30090 from williamhyun/SPARK-33179. Authored-by: William Hyun <williamhyun3@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
parent
e6c53c2c1b
commit
53783e706d
|
@ -638,7 +638,7 @@ def main():
|
||||||
else:
|
else:
|
||||||
# else we're running locally or Github Actions.
|
# else we're running locally or Github Actions.
|
||||||
build_tool = "sbt"
|
build_tool = "sbt"
|
||||||
hadoop_version = os.environ.get("HADOOP_PROFILE", "hadoop2.7")
|
hadoop_version = os.environ.get("HADOOP_PROFILE", "hadoop3.2")
|
||||||
hive_version = os.environ.get("HIVE_PROFILE", "hive2.3")
|
hive_version = os.environ.get("HIVE_PROFILE", "hive2.3")
|
||||||
if "GITHUB_ACTIONS" in os.environ:
|
if "GITHUB_ACTIONS" in os.environ:
|
||||||
test_env = "github_actions"
|
test_env = "github_actions"
|
||||||
|
|
Loading…
Reference in a new issue