b3a2cebc2b
### What changes were proposed in this pull request? Extend the `TransposeWindow` rule to transpose `Window` nodes, that have `Project` between them. ### Why are the changes needed? The analyzer will turn a `dataset.withColumn("colName", expressionWithWindowFunction)` method call to a `Project - Window - Project` chain in the logical plan. When this method is called multiple times in a row, then the projects can block the `Window` nodes from being transposed by the current `TransposeWindow` rule. TPCDS q47 and q57 are also improved by this. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? UT Closes #31980 from tanelk/SPARK-34807_transpose_window. Lead-authored-by: tanel.kiis@gmail.com <tanel.kiis@gmail.com> Co-authored-by: Tanel Kiis <tanel.kiis@gmail.com> Signed-off-by: Yuming Wang <yumwang@ebay.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |