spark-instrumented-optimizer/core/benchmarks/CoalescedRDDBenchmark-jdk11-results.txt

41 lines
4.2 KiB
Plaintext
Raw Normal View History

================================================================================================
Coalesced RDD , large scale
================================================================================================
[SPARK-34950][TESTS] Update benchmark results to the ones created by GitHub Actions machines ### What changes were proposed in this pull request? https://github.com/apache/spark/pull/32015 added a way to run benchmarks much more easily in the same GitHub Actions build. This PR updates the benchmark results by using the way. **NOTE** that looks like GitHub Actions use four types of CPU given my observations: - Intel(R) Xeon(R) Platinum 8171M CPU 2.60GHz - Intel(R) Xeon(R) CPU E5-2673 v4 2.30GHz - Intel(R) Xeon(R) CPU E5-2673 v3 2.40GHz - Intel(R) Xeon(R) Platinum 8272CL CPU 2.60GHz Given my quick research, seems like they perform roughly similarly: ![Screen Shot 2021-04-03 at 9 31 23 PM](https://user-images.githubusercontent.com/6477701/113478478-f4b57b80-94c3-11eb-9047-f81ca8c59672.png) I couldn't find enough information about Intel(R) Xeon(R) Platinum 8272CL CPU 2.60GHz but the performance seems roughly similar given the numbers. So shouldn't be a big deal especially given that this way is much easier, encourages contributors to run more and guarantee the same number of cores and same memory with the same softwares. ### Why are the changes needed? To have a base line of the benchmarks accordingly. ### Does this PR introduce _any_ user-facing change? No, dev-only. ### How was this patch tested? It was generated from: - [Run benchmarks: * (JDK 11)](https://github.com/HyukjinKwon/spark/actions/runs/713575465) - [Run benchmarks: * (JDK 8)](https://github.com/HyukjinKwon/spark/actions/runs/713154337) Closes #32044 from HyukjinKwon/SPARK-34950. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: Max Gekk <max.gekk@gmail.com>
2021-04-03 16:02:56 -04:00
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
Coalesced RDD: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
----------------------------------------------------------------------------------------------------------------------------
Coalesce Num Partitions: 100 Num Hosts: 1 365 454 104 0.3 3652.8 1.0X
Coalesce Num Partitions: 100 Num Hosts: 5 271 374 90 0.4 2708.0 1.3X
Coalesce Num Partitions: 100 Num Hosts: 10 265 337 88 0.4 2651.9 1.4X
Coalesce Num Partitions: 100 Num Hosts: 20 277 338 87 0.4 2772.8 1.3X
Coalesce Num Partitions: 100 Num Hosts: 40 313 320 6 0.3 3127.8 1.2X
Coalesce Num Partitions: 100 Num Hosts: 80 291 330 60 0.3 2909.5 1.3X
Coalesce Num Partitions: 500 Num Hosts: 1 974 1035 59 0.1 9741.6 0.4X
Coalesce Num Partitions: 500 Num Hosts: 5 443 450 6 0.2 4427.7 0.8X
Coalesce Num Partitions: 500 Num Hosts: 10 382 455 64 0.3 3816.6 1.0X
Coalesce Num Partitions: 500 Num Hosts: 20 359 402 39 0.3 3589.9 1.0X
Coalesce Num Partitions: 500 Num Hosts: 40 317 350 54 0.3 3165.5 1.2X
Coalesce Num Partitions: 500 Num Hosts: 80 267 337 81 0.4 2668.5 1.4X
Coalesce Num Partitions: 1000 Num Hosts: 1 1621 1712 84 0.1 16205.9 0.2X
Coalesce Num Partitions: 1000 Num Hosts: 5 582 638 85 0.2 5817.7 0.6X
Coalesce Num Partitions: 1000 Num Hosts: 10 398 446 74 0.3 3980.8 0.9X
Coalesce Num Partitions: 1000 Num Hosts: 20 332 410 108 0.3 3315.1 1.1X
Coalesce Num Partitions: 1000 Num Hosts: 40 345 377 54 0.3 3453.1 1.1X
Coalesce Num Partitions: 1000 Num Hosts: 80 294 353 69 0.3 2940.5 1.2X
Coalesce Num Partitions: 5000 Num Hosts: 1 6818 6906 151 0.0 68183.8 0.1X
Coalesce Num Partitions: 5000 Num Hosts: 5 1606 1719 138 0.1 16061.1 0.2X
Coalesce Num Partitions: 5000 Num Hosts: 10 965 1031 94 0.1 9653.8 0.4X
Coalesce Num Partitions: 5000 Num Hosts: 20 643 722 98 0.2 6427.4 0.6X
Coalesce Num Partitions: 5000 Num Hosts: 40 445 517 97 0.2 4448.8 0.8X
Coalesce Num Partitions: 5000 Num Hosts: 80 369 432 93 0.3 3688.6 1.0X
Coalesce Num Partitions: 10000 Num Hosts: 1 12850 12953 90 0.0 128503.9 0.0X
Coalesce Num Partitions: 10000 Num Hosts: 5 3431 3463 49 0.0 34307.1 0.1X
Coalesce Num Partitions: 10000 Num Hosts: 10 1687 1795 106 0.1 16874.8 0.2X
Coalesce Num Partitions: 10000 Num Hosts: 20 1084 1125 58 0.1 10844.2 0.3X
Coalesce Num Partitions: 10000 Num Hosts: 40 667 722 73 0.1 6674.4 0.5X
Coalesce Num Partitions: 10000 Num Hosts: 80 488 542 82 0.2 4883.9 0.7X