7eb6f45688
### What changes were proposed in this pull request? Change casting of map and struct values to strings by using the `{}` brackets instead of `[]`. The behavior is controlled by the SQL config `spark.sql.legacy.castComplexTypesToString.enabled`. When it is `true`, `CAST` wraps maps and structs by `[]` in casting to strings. Otherwise, if this is `false`, which is the default, maps and structs are wrapped by `{}`. ### Why are the changes needed? - To distinguish structs/maps from arrays. - To make `show`'s output consistent with Hive and conversions to Hive strings. - To display dataframe content in the same form by `spark-sql` and `show` - To be consistent with the `*.sql` tests ### Does this PR introduce _any_ user-facing change? Yes ### How was this patch tested? By existing test suite `CastSuite`. Closes #29308 from MaxGekk/show-struct-map. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> |
||
---|---|---|
.. | ||
linalg | ||
param | ||
tests | ||
__init__.py | ||
base.py | ||
classification.py | ||
clustering.py | ||
common.py | ||
evaluation.py | ||
feature.py | ||
fpm.py | ||
functions.py | ||
image.py | ||
pipeline.py | ||
recommendation.py | ||
regression.py | ||
stat.py | ||
tree.py | ||
tuning.py | ||
util.py | ||
wrapper.py |