d0c4d224e0
### What changes were proposed in this pull request?
A hive partition can have different `PartitionDesc`s from `TableDesc` for describing Serde/InputFormatClass/OutputFormatClass, for a hive partitioned table, we shall respect those in `PartitionDesc`.
### Why are the changes needed?
in many cases, that Spark reads hive tables could result in surprise because of this issue.
### Does this PR introduce _any_ user-facing change?
yes, hive partition table that contains different serde/input/output could be recognized by Spark
### How was this patch tested?
new test added
Closes #33406 from yaooqinn/SPARK-36197.
Authored-by: Kent Yao <yao@apache.org>
Signed-off-by: Kent Yao <yao@apache.org>
(cherry picked from commit
|
||
---|---|---|
.. | ||
benchmarks | ||
compatibility/src/test/scala/org/apache/spark/sql/hive/execution | ||
src | ||
pom.xml |