ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-27831][FOLLOWUP][SQL][TEST] ADDITIONAL_REMOTE_REPOSITORIES is a comma-delimited string

Browse source 
### What changes were proposed in this pull request?

This PR is a very minor follow-up to become robust because `spark.sql.additionalRemoteRepositories` is a configuration which has a comma-separated value.

### Why are the changes needed?

This makes sure that `getHiveContribJar` will not fail on the configuration changes.

### Does this PR introduce any user-facing change?

No.

### How was this patch tested?

Manual. Change the default value with multiple repositories and run the following.
```
build/sbt -Phive "project hive" "test-only org.apache.spark.sql.hive.HiveSparkSubmitSuite"
```

Closes #26096 from dongjoon-hyun/SPARK-27831.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Yuming Wang <wgyumg@gmail.com>
This commit is contained in:
Dongjoon Hyun 2019-10-12 18:47:28 -07:00 committed by Yuming Wang
parent d193248205
commit abba53e78b
1 changed files with 2 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
sql/hive/src/test/scala/org/apache/spark/sql/hive/test/TestHive.scala
View file

@ -649,12 +649,13 @@ private[sql] class TestHiveSessionStateBuilder(
}
private[hive] object HiveTestJars {
private val repository = SQLConf.ADDITIONAL_REMOTE_REPOSITORIES.defaultValueString
private val repository = SQLConf.ADDITIONAL_REMOTE_REPOSITORIES.defaultValueString.split(",")(0)
private val hiveTestJarsDir = Utils.createTempDir()
def getHiveContribJar(version: String = HiveUtils.builtinHiveVersion): File =
getJarFromUrl(s"${repository}org/apache/hive/hive-contrib/" +
s"$version/hive-contrib-$version.jar")
def getHiveHcatalogCoreJar(version: String = HiveUtils.builtinHiveVersion): File =
getJarFromUrl(s"${repository}org/apache/hive/hcatalog/hive-hcatalog-core/" +
s"$version/hive-hcatalog-core-$version.jar")