43dcb91a4c
## What changes were proposed in this pull request? Since Spark 2.2.0 ([SPARK-19678](https://issues.apache.org/jira/browse/SPARK-19678)), the below SQL changed from `broadcast join` to `sort merge join`: ```sql -- small external table with incorrect statistics CREATE EXTERNAL TABLE t1(c1 int) ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' WITH SERDEPROPERTIES ( 'serialization.format' = '1' ) STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat' LOCATION 'file:///tmp/t1' TBLPROPERTIES ( 'rawDataSize'='-1', 'numFiles'='0', 'totalSize'='0', 'COLUMN_STATS_ACCURATE'='false', 'numRows'='-1' ); -- big table CREATE TABLE t2 (c1 int) LOCATION 'file:///tmp/t2' TBLPROPERTIES ( 'rawDataSize'='23437737', 'numFiles'='12222', 'totalSize'='333442230', 'COLUMN_STATS_ACCURATE'='false', 'numRows'='443442223' ); explain SELECT t1.c1 FROM t1 INNER JOIN t2 ON t1.c1 = t2.c1; ``` This pr add a test case for this behavior change. ## How was this patch tested? unit tests Closes #24003 from wangyum/SPARK-19678. Authored-by: Yuming Wang <yumwang@ebay.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com> |
||
---|---|---|
.. | ||
benchmarks | ||
compatibility/src/test/scala/org/apache/spark/sql/hive/execution | ||
src | ||
pom.xml |