984ff396a2
### What changes were proposed in this pull request? In the PR, I propose to generate "stable" output attributes per the logical node of the `DESCRIBE TABLE` command. ### Why are the changes needed? This fixes the issue demonstrated by the example: ```scala val tbl = "testcat.ns1.ns2.tbl" sql(s"CREATE TABLE $tbl (c0 INT) USING _") val description = sql(s"DESCRIBE TABLE $tbl") description.drop("comment") ``` The `drop()` method fails with the error: ``` org.apache.spark.sql.AnalysisException: Resolved attribute(s) col_name#102,data_type#103 missing from col_name#29,data_type#30,comment#31 in operator !Project [col_name#102, data_type#103]. Attribute(s) with the same name appear in the operation: col_name,data_type. Please check if the right attribute(s) are used.; !Project [col_name#102, data_type#103] +- LocalRelation [col_name#29, data_type#30, comment#31] at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis(CheckAnalysis.scala:51) at org.apache.spark.sql.catalyst.analysis.CheckAnalysis.failAnalysis$(CheckAnalysis.scala:50) ``` ### Does this PR introduce _any_ user-facing change? Yes. After the changes, `drop()`/`add()` works as expected: ```scala description.drop("comment").show() +---------------+---------+ | col_name|data_type| +---------------+---------+ | c0| int| | | | | # Partitioning| | |Not partitioned| | +---------------+---------+ ``` ### How was this patch tested? 1. Run new test: ``` $ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *DataSourceV2SQLSuite" ``` 2. Run existing test suite: ``` $ build/sbt -Phive-2.3 -Phive-thriftserver "test:testOnly *CatalogedDDLSuite" ``` Closes #31676 from MaxGekk/describe-table-drop-column. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |