[SPARK-24312][SQL] Upgrade to 2.3.3 for Hive Metastore Client 2.3
## What changes were proposed in this pull request? Hive 2.3.3 was [released on April 3rd](https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12342162&styleName=Text&projectId=12310843). This PR aims to upgrade Hive Metastore Client 2.3 from 2.3.2 to 2.3.3. ## How was this patch tested? Pass the Jenkins with the existing tests. Author: Dongjoon Hyun <dongjoon@apache.org> Closes #21359 from dongjoon-hyun/SPARK-24312.
This commit is contained in:
parent
1c4553d67d
commit
7f82c4a47e
|
@ -1214,7 +1214,7 @@ The following options can be used to configure the version of Hive that is used
|
||||||
<td><code>1.2.1</code></td>
|
<td><code>1.2.1</code></td>
|
||||||
<td>
|
<td>
|
||||||
Version of the Hive metastore. Available
|
Version of the Hive metastore. Available
|
||||||
options are <code>0.12.0</code> through <code>2.3.2</code>.
|
options are <code>0.12.0</code> through <code>2.3.3</code>.
|
||||||
</td>
|
</td>
|
||||||
</tr>
|
</tr>
|
||||||
<tr>
|
<tr>
|
||||||
|
@ -2237,7 +2237,7 @@ referencing a singleton.
|
||||||
Spark SQL is designed to be compatible with the Hive Metastore, SerDes and UDFs.
|
Spark SQL is designed to be compatible with the Hive Metastore, SerDes and UDFs.
|
||||||
Currently, Hive SerDes and UDFs are based on Hive 1.2.1,
|
Currently, Hive SerDes and UDFs are based on Hive 1.2.1,
|
||||||
and Spark SQL can be connected to different versions of Hive Metastore
|
and Spark SQL can be connected to different versions of Hive Metastore
|
||||||
(from 0.12.0 to 2.3.2. Also see [Interacting with Different Versions of Hive Metastore](#interacting-with-different-versions-of-hive-metastore)).
|
(from 0.12.0 to 2.3.3. Also see [Interacting with Different Versions of Hive Metastore](#interacting-with-different-versions-of-hive-metastore)).
|
||||||
|
|
||||||
#### Deploying in Existing Hive Warehouses
|
#### Deploying in Existing Hive Warehouses
|
||||||
|
|
||||||
|
|
|
@ -62,7 +62,7 @@ private[spark] object HiveUtils extends Logging {
|
||||||
|
|
||||||
val HIVE_METASTORE_VERSION = buildConf("spark.sql.hive.metastore.version")
|
val HIVE_METASTORE_VERSION = buildConf("spark.sql.hive.metastore.version")
|
||||||
.doc("Version of the Hive metastore. Available options are " +
|
.doc("Version of the Hive metastore. Available options are " +
|
||||||
s"<code>0.12.0</code> through <code>2.3.2</code>.")
|
s"<code>0.12.0</code> through <code>2.3.3</code>.")
|
||||||
.stringConf
|
.stringConf
|
||||||
.createWithDefault(builtinHiveVersion)
|
.createWithDefault(builtinHiveVersion)
|
||||||
|
|
||||||
|
|
|
@ -98,7 +98,7 @@ private[hive] object IsolatedClientLoader extends Logging {
|
||||||
case "2.0" | "2.0.0" | "2.0.1" => hive.v2_0
|
case "2.0" | "2.0.0" | "2.0.1" => hive.v2_0
|
||||||
case "2.1" | "2.1.0" | "2.1.1" => hive.v2_1
|
case "2.1" | "2.1.0" | "2.1.1" => hive.v2_1
|
||||||
case "2.2" | "2.2.0" => hive.v2_2
|
case "2.2" | "2.2.0" => hive.v2_2
|
||||||
case "2.3" | "2.3.0" | "2.3.1" | "2.3.2" => hive.v2_3
|
case "2.3" | "2.3.0" | "2.3.1" | "2.3.2" | "2.3.3" => hive.v2_3
|
||||||
}
|
}
|
||||||
|
|
||||||
private def downloadVersion(
|
private def downloadVersion(
|
||||||
|
|
|
@ -75,7 +75,7 @@ package object client {
|
||||||
exclusions = Seq("org.apache.curator:*",
|
exclusions = Seq("org.apache.curator:*",
|
||||||
"org.pentaho:pentaho-aggdesigner-algorithm"))
|
"org.pentaho:pentaho-aggdesigner-algorithm"))
|
||||||
|
|
||||||
case object v2_3 extends HiveVersion("2.3.2",
|
case object v2_3 extends HiveVersion("2.3.3",
|
||||||
exclusions = Seq("org.apache.curator:*",
|
exclusions = Seq("org.apache.curator:*",
|
||||||
"org.pentaho:pentaho-aggdesigner-algorithm"))
|
"org.pentaho:pentaho-aggdesigner-algorithm"))
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue