ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-28532][SQL] Make optimizer batch "subquery" FixedPoint(1)

Browse source 
## What changes were proposed in this pull request?
In the Catalyst optimizer, the batch subquery actually calls the optimizer recursively. Therefore it makes no sense to enforce idempotence on it and we change this batch to `FixedPoint(1)`.

## How was this patch tested?
Existing UTs.

Closes #25267 from yeshengm/SPARK-28532.

Authored-by: Yesheng Ma <kimi.ysma@gmail.com>
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
This commit is contained in:
Yesheng Ma 2019-07-26 22:48:42 -07:00 committed by gatorsmile
parent 558dd23601
commit e037a11494
1 changed files with 1 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala
View file

@ -50,7 +50,6 @@ abstract class Optimizer(sessionCatalog: SessionCatalog)
override protected val blacklistedOnceBatches: Set[String] = override protected val blacklistedOnceBatches: Set[String] =
Set("Pullup Correlated Expressions", Set("Pullup Correlated Expressions",
"Join Reorder", "Join Reorder",
"Subquery",
"Extract Python UDFs" "Extract Python UDFs"
) )
@ -156,7 +155,7 @@ abstract class Optimizer(sessionCatalog: SessionCatalog)
PropagateEmptyRelation) :: PropagateEmptyRelation) ::
Batch("Pullup Correlated Expressions", Once, Batch("Pullup Correlated Expressions", Once,
PullupCorrelatedPredicates) :: PullupCorrelatedPredicates) ::
Batch("Subquery", Once, Batch("Subquery", FixedPoint(1),
OptimizeSubqueries) :: OptimizeSubqueries) ::
Batch("Replace Operators", fixedPoint, Batch("Replace Operators", fixedPoint,
RewriteExceptAll, RewriteExceptAll,