[SPARK-7686] [SQL] DescribeCommand is assigned wrong output attributes in SparkStrategies
In `SparkStrategies`, `RunnableDescribeCommand` is called with the output attributes of the table being described rather than the attributes for the `describe` command's output. I discovered this issue because it caused type conversion errors in some UnsafeRow conversion code that I'm writing.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #6217 from JoshRosen/SPARK-7686 and squashes the following commits:
953a344 [Josh Rosen] Fix SPARK-7686 with a simple change in SparkStrategies.
a4eec9f [Josh Rosen] Add failing regression test for SPARK-7686
(cherry picked from commit 564562874f
)
Signed-off-by: Reynold Xin <rxin@databricks.com>
This commit is contained in:
parent
6df71eb8c1
commit
53d6ab51b2
|
@ -354,10 +354,10 @@ private[sql] abstract class SparkStrategies extends QueryPlanner[SparkPlan] {
|
|||
case c: CreateTableUsingAsSelect if !c.temporary =>
|
||||
sys.error("Tables created with SQLContext must be TEMPORARY. Use a HiveContext instead.")
|
||||
|
||||
case LogicalDescribeCommand(table, isExtended) =>
|
||||
case describe @ LogicalDescribeCommand(table, isExtended) =>
|
||||
val resultPlan = self.sqlContext.executePlan(table).executedPlan
|
||||
ExecutedCommand(
|
||||
RunnableDescribeCommand(resultPlan, resultPlan.output, isExtended)) :: Nil
|
||||
RunnableDescribeCommand(resultPlan, describe.output, isExtended)) :: Nil
|
||||
|
||||
case _ => Nil
|
||||
}
|
||||
|
|
|
@ -99,4 +99,10 @@ class DDLTestSuite extends DataSourceTest {
|
|||
Row("arrayType", "array<string>", ""),
|
||||
Row("structType", "struct<f1:string,f2:int>", "")
|
||||
))
|
||||
|
||||
test("SPARK-7686 DescribeCommand should have correct physical plan output attributes") {
|
||||
val attributes = sql("describe ddlPeople").queryExecution.executedPlan.output
|
||||
assert(attributes.map(_.name) === Seq("col_name", "data_type", "comment"))
|
||||
assert(attributes.map(_.dataType).toSet === Set(StringType))
|
||||
}
|
||||
}
|
||||
|
|
Loading…
Reference in a new issue