spark-instrumented-optimizer/sql/hive
turbofei 717ec5e9e3 [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map parsed from partition path should be case insensitive
### What changes were proposed in this pull request?

This is a follow up of https://github.com/apache/spark/pull/25979.
When we inserting overwrite  an external hive partitioned table with upper case dynamic partition key, exception thrown.

like:
```
org.apache.spark.SparkException: Dynamic partition key P1 is not among written partition paths.
```
The root cause is that Hive metastore is not case preserving and keeps partition columns with lower cased names, see details in:

ddd8d5f5a0/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala (L895-L901)
e28914095a/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/InsertIntoHiveTable.scala (L228-L234)

In this PR, we convert the dynamic partition map to a case insensitive map.
### Why are the changes needed?

To fix the issue when inserting overwrite into external hive partitioned table with upper case dynamic partition key.

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
UT.

Closes #28765 from turboFei/SPARK-29295-follow-up.

Authored-by: turbofei <fwang12@ebay.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2020-06-09 15:57:18 +00:00
..
benchmarks [SPARK-31684][SQL] Overwrite partition failed with 'WRONG FS' when the target partition is not belong to the filesystem as same as the table 2020-05-19 14:08:51 +00:00
compatibility/src/test/scala/org/apache/spark/sql/hive/execution [SPARK-31707][SQL] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax 2020-05-17 02:27:23 +00:00
src [SPARK-29295][SQL][FOLLOWUP] Dynamic partition map parsed from partition path should be case insensitive 2020-06-09 15:57:18 +00:00
pom.xml [SPARK-30950][BUILD] Setting version to 3.1.0-SNAPSHOT 2020-02-25 19:44:31 -08:00