spark-instrumented-optimizer/sql/core
Wenchen Fan 1f3863c856 [SPARK-29759][SQL] LocalShuffleReaderExec.outputPartitioning should use the corrected attributes
### What changes were proposed in this pull request?

Update `LocalShuffleReaderExec.outputPartitioning` to use attributes from `ReusedQueryStage`.

This also removes the override `doCanonicalize` in local/coalesced shuffle reader, as these 2 operators change the output partitioning. It's not safe to strip them in the canonicalized query plan.

### Why are the changes needed?

We will have an invalid output partitioning if we don fix it.

### Does this PR introduce any user-facing change?

no

### How was this patch tested?

existing tests

Closes #26400 from cloud-fan/aqe.

Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Xiao Li <gatorsmile@gmail.com>
2019-11-06 14:33:52 -08:00
..
benchmarks [SPARK-29562][SQL] Speed up and slim down metric aggregation in SQL listener 2019-10-24 22:18:10 -07:00
src [SPARK-29759][SQL] LocalShuffleReaderExec.outputPartitioning should use the corrected attributes 2019-11-06 14:33:52 -08:00
v1.2.1/src [SPARK-29277][SQL][test-hadoop3.2] Add early DSv2 filter and projection pushdown 2019-10-31 08:25:32 -07:00
v2.3.5/src [SPARK-29277][SQL][test-hadoop3.2] Add early DSv2 filter and projection pushdown 2019-10-31 08:25:32 -07:00
pom.xml Revert "Prepare Spark release v3.0.0-preview-rc2" 2019-10-30 17:45:44 -07:00