Gengliang Wang 94fc0e3235 [SPARK-30428][SQL] File source V2: support partition pruning ... ### What changes were proposed in this pull request? File source V2: support partition pruning. Note: subquery predicates are not pushed down for partition pruning even after this PR, due to the limitation for the current data source V2 API and framework. The rule `PlanSubqueries` requires the subquery expression to be in the children or class parameters in `SparkPlan`, while the condition is not satisfied for `BatchScanExec`. ### Why are the changes needed? It's important for reading performance. ### Does this PR introduce any user-facing change? No ### How was this patch tested? New unit tests for all the V2 file sources Closes #27112 from gengliangwang/PartitionPruningInFileScan. Authored-by: Gengliang Wang <gengliang.wang@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>		2020-01-09 21:53:37 +08:00
..
main	[SPARK-29981][BUILD][FOLLOWUP] Change hive.version.short	2019-11-23 12:50:50 -08:00
test/scala/org/apache/spark/sql/execution/datasources/orc	[SPARK-30428][SQL] File source V2: support partition pruning	2020-01-09 21:53:37 +08:00