spark-instrumented-optimizer/sql/catalyst/src/main
Jungtaek Lim (HeartSaVioR) dea18ee85b [SPARK-22000][SQL] Address missing Upcast in JavaTypeInference.deserializerFor
## What changes were proposed in this pull request?

Spark expects the type of column and the type of matching field is same when deserializing to Object, but Spark hasn't actually restrict it (at least for Java bean encoder) and some users just do it and experience undefined behavior (in SPARK-22000, Spark throws compilation failure on generated code because it calls `.toString()` against primitive type.

It doesn't produce error in Scala side because `ScalaReflection.deserializerFor` properly inject Upcast if necessary. This patch proposes applying same thing to `JavaTypeInference.deserializerFor` as well.

Credit to srowen, maropu, and cloud-fan since they provided various approaches to solve this.

## How was this patch tested?

Added UT which query is slightly modified based on sample code in attachment on JIRA issue.

Closes #23854 from HeartSaVioR/SPARK-22000.

Authored-by: Jungtaek Lim (HeartSaVioR) <kabhwan@gmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2019-02-27 13:47:20 +08:00
..
antlr4/org/apache/spark/sql/catalyst/parser [SPARK-26215][SQL] Define reserved/non-reserved keywords based on the ANSI SQL standard 2019-02-23 08:38:47 +09:00
java/org/apache/spark/sql [SPARK-26621][CORE] Use ConfigEntry for hardcoded configs for shuffle categories. 2019-01-17 12:29:17 -06:00
scala/org/apache/spark/sql [SPARK-22000][SQL] Address missing Upcast in JavaTypeInference.deserializerFor 2019-02-27 13:47:20 +08:00