[SPARK-36020][SQL][FOLLOWUP] RemoveRedundantProjects should retain the LOGICAL_PLAN_TAG tag
### What changes were proposed in this pull request? This is a followup of https://github.com/apache/spark/pull/33222 . https://github.com/apache/spark/pull/33222 made a mistake that, `RemoveRedundantProjects` may lose the `LOGICAL_PLAN_TAG` tag, even though the logical plan link is retained. This was actually caught by the test `LogicalPlanTagInSparkPlanSuite`, but was not being taken care of. There is no problem so far, but losing information can always lead to potential bugs. ### Why are the changes needed? fix a mistake ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? existing test Closes #33442 from cloud-fan/minor. Authored-by: Wenchen Fan <wenchen@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
This commit is contained in:
parent
efcce23b91
commit
94aece4325
|
@ -49,7 +49,11 @@ object RemoveRedundantProjects extends Rule[SparkPlan] {
|
|||
plan match {
|
||||
case p @ ProjectExec(_, child) =>
|
||||
if (isRedundant(p, child, requireOrdering) && canRemove(p, child)) {
|
||||
removeProject(child, requireOrdering)
|
||||
val newPlan = removeProject(child, requireOrdering)
|
||||
// The `newPlan` should retain the logical plan link already. We call `setLogicalLink`
|
||||
// here to make sure the `newPlan` sets the `LOGICAL_PLAN_TAG` tag.
|
||||
newPlan.setLogicalLink(child.logicalLink.get)
|
||||
newPlan
|
||||
} else {
|
||||
p.mapChildren(removeProject(_, false))
|
||||
}
|
||||
|
|
|
@ -131,7 +131,7 @@ class LogicalPlanTagInSparkPlanSuite extends TPCDSQuerySuite with DisableAdaptiv
|
|||
}
|
||||
|
||||
private def getLogicalPlan(node: SparkPlan): LogicalPlan = {
|
||||
node.logicalLink.getOrElse {
|
||||
node.getTagValue(SparkPlan.LOGICAL_PLAN_TAG).getOrElse {
|
||||
fail(node.getClass.getSimpleName + " does not have a logical plan link")
|
||||
}
|
||||
}
|
||||
|
|
Loading…
Reference in a new issue