ebf01ec3c1
### What changes were proposed in this pull request? https://github.com/apache/spark/pull/32015 added a way to run benchmarks much more easily in the same GitHub Actions build. This PR updates the benchmark results by using the way. **NOTE** that looks like GitHub Actions use four types of CPU given my observations: - Intel(R) Xeon(R) Platinum 8171M CPU 2.60GHz - Intel(R) Xeon(R) CPU E5-2673 v4 2.30GHz - Intel(R) Xeon(R) CPU E5-2673 v3 2.40GHz - Intel(R) Xeon(R) Platinum 8272CL CPU 2.60GHz Given my quick research, seems like they perform roughly similarly:  I couldn't find enough information about Intel(R) Xeon(R) Platinum 8272CL CPU 2.60GHz but the performance seems roughly similar given the numbers. So shouldn't be a big deal especially given that this way is much easier, encourages contributors to run more and guarantee the same number of cores and same memory with the same softwares. ### Why are the changes needed? To have a base line of the benchmarks accordingly. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? It was generated from: - [Run benchmarks: * (JDK 11)](https://github.com/HyukjinKwon/spark/actions/runs/713575465) - [Run benchmarks: * (JDK 8)](https://github.com/HyukjinKwon/spark/actions/runs/713154337) Closes #32044 from HyukjinKwon/SPARK-34950. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: Max Gekk <max.gekk@gmail.com>
41 lines
4.2 KiB
Plaintext
41 lines
4.2 KiB
Plaintext
================================================================================================
|
|
Coalesced RDD , large scale
|
|
================================================================================================
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz
|
|
Coalesced RDD: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
----------------------------------------------------------------------------------------------------------------------------
|
|
Coalesce Num Partitions: 100 Num Hosts: 1 394 423 25 0.3 3942.7 1.0X
|
|
Coalesce Num Partitions: 100 Num Hosts: 5 317 339 23 0.3 3171.6 1.2X
|
|
Coalesce Num Partitions: 100 Num Hosts: 10 310 324 20 0.3 3095.9 1.3X
|
|
Coalesce Num Partitions: 100 Num Hosts: 20 285 292 8 0.4 2849.0 1.4X
|
|
Coalesce Num Partitions: 100 Num Hosts: 40 271 292 22 0.4 2712.7 1.5X
|
|
Coalesce Num Partitions: 100 Num Hosts: 80 287 300 18 0.3 2874.2 1.4X
|
|
Coalesce Num Partitions: 500 Num Hosts: 1 839 872 33 0.1 8388.1 0.5X
|
|
Coalesce Num Partitions: 500 Num Hosts: 5 389 395 5 0.3 3892.4 1.0X
|
|
Coalesce Num Partitions: 500 Num Hosts: 10 352 356 3 0.3 3522.0 1.1X
|
|
Coalesce Num Partitions: 500 Num Hosts: 20 315 322 11 0.3 3145.0 1.3X
|
|
Coalesce Num Partitions: 500 Num Hosts: 40 304 324 22 0.3 3036.4 1.3X
|
|
Coalesce Num Partitions: 500 Num Hosts: 80 286 301 23 0.4 2855.7 1.4X
|
|
Coalesce Num Partitions: 1000 Num Hosts: 1 1390 1422 28 0.1 13898.0 0.3X
|
|
Coalesce Num Partitions: 1000 Num Hosts: 5 535 562 26 0.2 5347.4 0.7X
|
|
Coalesce Num Partitions: 1000 Num Hosts: 10 419 425 8 0.2 4193.8 0.9X
|
|
Coalesce Num Partitions: 1000 Num Hosts: 20 358 364 9 0.3 3575.8 1.1X
|
|
Coalesce Num Partitions: 1000 Num Hosts: 40 323 329 5 0.3 3234.8 1.2X
|
|
Coalesce Num Partitions: 1000 Num Hosts: 80 306 316 11 0.3 3060.3 1.3X
|
|
Coalesce Num Partitions: 5000 Num Hosts: 1 5967 6744 1289 0.0 59666.6 0.1X
|
|
Coalesce Num Partitions: 5000 Num Hosts: 5 2010 2032 19 0.0 20104.1 0.2X
|
|
Coalesce Num Partitions: 5000 Num Hosts: 10 1132 1161 25 0.1 11324.9 0.3X
|
|
Coalesce Num Partitions: 5000 Num Hosts: 20 687 714 39 0.1 6874.2 0.6X
|
|
Coalesce Num Partitions: 5000 Num Hosts: 40 507 511 3 0.2 5070.1 0.8X
|
|
Coalesce Num Partitions: 5000 Num Hosts: 80 375 383 12 0.3 3749.7 1.1X
|
|
Coalesce Num Partitions: 10000 Num Hosts: 1 10782 10833 63 0.0 107816.6 0.0X
|
|
Coalesce Num Partitions: 10000 Num Hosts: 5 3819 3901 72 0.0 38185.3 0.1X
|
|
Coalesce Num Partitions: 10000 Num Hosts: 10 2021 2030 12 0.0 20212.9 0.2X
|
|
Coalesce Num Partitions: 10000 Num Hosts: 20 1138 1168 44 0.1 11378.0 0.3X
|
|
Coalesce Num Partitions: 10000 Num Hosts: 40 716 723 9 0.1 7157.2 0.6X
|
|
Coalesce Num Partitions: 10000 Num Hosts: 80 504 514 9 0.2 5039.9 0.8X
|
|
|
|
|