6d69068057
### What changes were proposed in this pull request? Make WithFields Expression not foldable. ### Why are the changes needed? The following query currently fails on master brach: ``` sql("SELECT named_struct('a', 1, 'b', 2) a") .select($"a".withField("c", lit(3)).as("a")) .show(false) // java.lang.UnsupportedOperationException: Cannot evaluate expression: with_fields(named_struct(a, 1, b, 2), c, 3) ``` This happens because the Catalyst optimizer sees that the WithFields Expression is foldable and tries to statically evaluate the WithFields Expression (via the ConstantFolding rule), however it cannot do so because WithFields Expression is Unevaluable. ### Does this PR introduce _any_ user-facing change? Yes, queries like the one shared above will now succeed. That said, this bug was introduced in Spark 3.1.0 which has yet to be released. ### How was this patch tested? A new unit test was added. Closes #29338 from fqaiser94/SPARK-32521. Lead-authored-by: fqaiser94@gmail.com <fqaiser94@gmail.com> Co-authored-by: fqaiser94 <fqaiser94@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |