spark-instrumented-optimizer/sql/catalyst
Davies Liu 07f92ef1fa [SPARK-13376] [SPARK-13476] [SQL] improve column pruning
## What changes were proposed in this pull request?

This PR mostly rewrite the ColumnPruning rule to support most of the SQL logical plans (except those for Dataset).

This PR also fix a bug in Generate, it should always output UnsafeRow, added an regression test for that.

## How was this patch tested?

This is test by unit tests, also manually test with TPCDS Q78, which could prune all unused columns successfully, improved the performance by 78% (from 22s to 12s).

Author: Davies Liu <davies@databricks.com>

Closes #11354 from davies/fix_column_pruning.
2016-02-25 00:13:07 -08:00
..
src [SPARK-13376] [SPARK-13476] [SQL] improve column pruning 2016-02-25 00:13:07 -08:00
pom.xml [SPARK-11565] Replace deprecated DigestUtils.shaHex call 2016-02-10 09:52:35 +00:00