313f3c5460
### What changes were proposed in this pull request? RemoveRedundantAliases may change DataWritingCommand's parameter's attribute name. In the UT's case before RemoveRedundantAliases the partitionColumns is `CAL_DT`, and change by RemoveRedundantAliases and change to `cal_dt` then case the error case ### Why are the changes needed? Fix bug ### Does this PR introduce _any_ user-facing change? For below SQL case ``` sql("create table t1(cal_dt date) using parquet") sql("insert into t1 values (date'2021-06-27'),(date'2021-06-28'),(date'2021-06-29'),(date'2021-06-30')") sql("create view t1_v as select * from t1") sql("CREATE TABLE t2 USING PARQUET PARTITIONED BY (CAL_DT) AS SELECT 1 AS FLAG,CAL_DT FROM t1_v WHERE CAL_DT BETWEEN '2021-06-27' AND '2021-06-28'") sql("INSERT INTO t2 SELECT 2 AS FLAG,CAL_DT FROM t1_v WHERE CAL_DT BETWEEN '2021-06-29' AND '2021-06-30'") ``` Before this pr ``` sql("SELECT * FROM t2 WHERE CAL_DT BETWEEN '2021-06-29' AND '2021-06-30'").show +----+------+ |FLAG|CAL_DT| +----+------+ +----+------+ sql("SELECT * FROM t2 ").show +----+----------+ |FLAG| CAL_DT| +----+----------+ | 1|2021-06-27| | 1|2021-06-28| +----+----------+ ``` After this pr ``` sql("SELECT * FROM t2 WHERE CAL_DT BETWEEN '2021-06-29' AND '2021-06-30'").show +----+------+ |FLAG|CAL_DT| +----+------+ | 2|2021-06-29| | 2|2021-06-30| +----+------+ sql("SELECT * FROM t2 ").show +----+----------+ |FLAG| CAL_DT| +----+----------+ | 1|2021-06-27| | 1|2021-06-28| | 2|2021-06-29| | 2|2021-06-30| +----+----------+ ``` ### How was this patch tested? Added UT Closes #33324 from AngersZhuuuu/SPARK-36093. Authored-by: Angerszhuuuu <angers.zhu@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |