c73f70bb0d
### What changes were proposed in this pull request? Rewrote one `ExtractGenerator` case such that it would not rely on a side effect of the flatmap function. ### Why are the changes needed? With the dataframe api it is possible to have a lazy sequence as the `output` of a `LogicalPlan`. When exploding a column on this dataframe using the `withColumn("newName", explode(col("name")))` method, the `ExtractGenerator` does not extract the generator and `CheckAnalysis` would throw an exception. ### Does this PR introduce _any_ user-facing change? Bugfix Before this, the work around was to put `.select("*")` before the explode. ### How was this patch tested? UT Closes #31213 from tanelk/SPARK-34141_extract_generator. Authored-by: tanel.kiis@gmail.com <tanel.kiis@gmail.com> Signed-off-by: Sean Owen <srowen@gmail.com> |
||
---|---|---|
.. | ||
benchmarks | ||
src | ||
pom.xml |