ebf01ec3c1
### What changes were proposed in this pull request? https://github.com/apache/spark/pull/32015 added a way to run benchmarks much more easily in the same GitHub Actions build. This PR updates the benchmark results by using the way. **NOTE** that looks like GitHub Actions use four types of CPU given my observations: - Intel(R) Xeon(R) Platinum 8171M CPU 2.60GHz - Intel(R) Xeon(R) CPU E5-2673 v4 2.30GHz - Intel(R) Xeon(R) CPU E5-2673 v3 2.40GHz - Intel(R) Xeon(R) Platinum 8272CL CPU 2.60GHz Given my quick research, seems like they perform roughly similarly:  I couldn't find enough information about Intel(R) Xeon(R) Platinum 8272CL CPU 2.60GHz but the performance seems roughly similar given the numbers. So shouldn't be a big deal especially given that this way is much easier, encourages contributors to run more and guarantee the same number of cores and same memory with the same softwares. ### Why are the changes needed? To have a base line of the benchmarks accordingly. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? It was generated from: - [Run benchmarks: * (JDK 11)](https://github.com/HyukjinKwon/spark/actions/runs/713575465) - [Run benchmarks: * (JDK 8)](https://github.com/HyukjinKwon/spark/actions/runs/713154337) Closes #32044 from HyukjinKwon/SPARK-34950. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: Max Gekk <max.gekk@gmail.com>
61 lines
4.6 KiB
Plaintext
61 lines
4.6 KiB
Plaintext
================================================================================================
|
|
Parquet writer benchmark
|
|
================================================================================================
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
|
|
Parquet writer benchmark: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Output Single Int Column 2067 2087 29 7.6 131.4 1.0X
|
|
Output Single Double Column 1977 2013 52 8.0 125.7 1.0X
|
|
Output Int and String Column 7387 7410 33 2.1 469.7 0.3X
|
|
Output Partitions 3720 3831 156 4.2 236.5 0.6X
|
|
Output Buckets 4987 5028 57 3.2 317.1 0.4X
|
|
|
|
|
|
================================================================================================
|
|
ORC writer benchmark
|
|
================================================================================================
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
|
|
ORC writer benchmark: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Output Single Int Column 1264 1284 29 12.4 80.3 1.0X
|
|
Output Single Double Column 1708 1711 4 9.2 108.6 0.7X
|
|
Output Int and String Column 5988 6019 43 2.6 380.7 0.2X
|
|
Output Partitions 3192 3269 108 4.9 203.0 0.4X
|
|
Output Buckets 4439 4472 47 3.5 282.2 0.3X
|
|
|
|
|
|
================================================================================================
|
|
JSON writer benchmark
|
|
================================================================================================
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
|
|
JSON writer benchmark: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Output Single Int Column 2187 2194 10 7.2 139.0 1.0X
|
|
Output Single Double Column 2808 2833 35 5.6 178.5 0.8X
|
|
Output Int and String Column 6190 6224 49 2.5 393.5 0.4X
|
|
Output Partitions 3646 3705 83 4.3 231.8 0.6X
|
|
Output Buckets 4891 5325 614 3.2 311.0 0.4X
|
|
|
|
|
|
================================================================================================
|
|
CSV writer benchmark
|
|
================================================================================================
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
|
|
CSV writer benchmark: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------------------------------
|
|
Output Single Int Column 3162 3168 9 5.0 201.0 1.0X
|
|
Output Single Double Column 3647 3660 19 4.3 231.9 0.9X
|
|
Output Int and String Column 7266 7271 6 2.2 462.0 0.4X
|
|
Output Partitions 4857 4930 103 3.2 308.8 0.7X
|
|
Output Buckets 6809 6897 124 2.3 432.9 0.5X
|
|
|
|
|