spark-instrumented-optimizer/sql/core
Ala Luszczak 36902e10c6 [SPARK-26878] QueryTest.compare() does not handle maps with array keys correctly
## What changes were proposed in this pull request?

The previous strategy for comparing Maps leveraged sorting (key, value) tuples by their _.toString. However, the _.toString representation of an arrays has nothing to do with it's content. If a map has array keys, it's (key, value) pairs would be compared with other maps essentially at random. This could results in false negatives in tests.

This changes first compares keys together to find the matching ones, and then compares associated values.

## How was this patch tested?

New unit test added.

Closes #23789 from ala/compare-map.

Authored-by: Ala Luszczak <ala@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2019-02-18 10:39:31 +08:00
..
benchmarks [SPARK-26745][SQL] Revert count optimization in JSON datasource by SPARK-24959 2019-01-31 14:32:31 +08:00
src [SPARK-26878] QueryTest.compare() does not handle maps with array keys correctly 2019-02-18 10:39:31 +08:00
pom.xml [SPARK-25956] Make Scala 2.12 as default Scala version in Spark 3.0 2018-11-14 16:22:23 -08:00