b0c2b3bfd9
## What changes were proposed in this pull request? We only supported nested schema pruning for Parquet previously. This proposes to support nested schema pruning for ORC too. Note: This only covers ORC v1. For ORC v2, the necessary change is at the schema pruning rule. We should deal with ORC v2 as a TODO item, in order to reduce review burden. ## How was this patch tested? Added tests. Closes #23943 from viirya/nested-schema-pruning-orc. Authored-by: Liang-Chi Hsieh <viirya@gmail.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
41 lines
3.1 KiB
Plaintext
41 lines
3.1 KiB
Plaintext
================================================================================================
|
|
Nested Schema Pruning Benchmark For ORC v1
|
|
================================================================================================
|
|
|
|
Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
|
|
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
|
|
Selection: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Top-level column 116 151 36 8.6 116.3 1.0X
|
|
Nested column 544 604 31 1.8 544.5 0.2X
|
|
|
|
Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
|
|
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
|
|
Limiting: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Top-level column 360 397 32 2.8 360.4 1.0X
|
|
Nested column 3322 3503 166 0.3 3322.4 0.1X
|
|
|
|
Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
|
|
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
|
|
Repartitioning: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Top-level column 292 334 32 3.4 291.8 1.0X
|
|
Nested column 3306 3489 123 0.3 3305.7 0.1X
|
|
|
|
Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
|
|
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
|
|
Repartitioning by exprs: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Top-level column 302 333 27 3.3 302.0 1.0X
|
|
Nested column 2697 3347 390 0.4 2697.4 0.1X
|
|
|
|
Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.3
|
|
Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz
|
|
Sorting: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Top-level column 316 440 146 3.2 315.8 1.0X
|
|
Nested column 2728 2928 205 0.4 2727.9 0.1X
|
|
|
|
|