ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/sql/core/benchmarks/CompressionSchemeBenchmark-jdk11-results.txt
HyukjinKwon ebf01ec3c1 [SPARK-34950][TESTS] Update benchmark results to the ones created by GitHub Actions machines 
### What changes were proposed in this pull request?

https://github.com/apache/spark/pull/32015 added a way to run benchmarks much more easily in the same GitHub Actions build. This PR updates the benchmark results by using the way.

**NOTE** that looks like GitHub Actions use four types of CPU given my observations:

- Intel(R) Xeon(R) Platinum 8171M CPU  2.60GHz
- Intel(R) Xeon(R) CPU E5-2673 v4  2.30GHz
- Intel(R) Xeon(R) CPU E5-2673 v3  2.40GHz
- Intel(R) Xeon(R) Platinum 8272CL CPU  2.60GHz

Given my quick research, seems like they perform roughly similarly:

![Screen Shot 2021-04-03 at 9 31 23 PM](https://user-images.githubusercontent.com/6477701/113478478-f4b57b80-94c3-11eb-9047-f81ca8c59672.png)

I couldn't find enough information about Intel(R) Xeon(R) Platinum 8272CL CPU  2.60GHz but the performance seems roughly similar given the numbers.

So shouldn't be a big deal especially given that this way is much easier, encourages contributors to run more and guarantee the same number of cores and same memory with the same softwares.

### Why are the changes needed?

To have a base line of the benchmarks accordingly.

### Does this PR introduce _any_ user-facing change?

No, dev-only.

### How was this patch tested?

It was generated from:

- [Run benchmarks: * (JDK 11)](https://github.com/HyukjinKwon/spark/actions/runs/713575465)
- [Run benchmarks: * (JDK 8)](https://github.com/HyukjinKwon/spark/actions/runs/713154337)

Closes #32044 from HyukjinKwon/SPARK-34950.

Authored-by: HyukjinKwon <gurwls223@apache.org>
Signed-off-by: Max Gekk <max.gekk@gmail.com>
2021-04-03 23:02:56 +03:00

138 lines
12 KiB
Plaintext
Raw Blame History

================================================================================================
Compression Scheme Benchmark
================================================================================================
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
BOOLEAN Encode: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough(1.000) 5 6 0 13175.1 0.1 1.0X
RunLengthEncoding(2.498) 907 920 11 74.0 13.5 0.0X
BooleanBitSet(0.125) 594 609 11 113.1 8.8 0.0X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
BOOLEAN Decode: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough 326 341 20 205.8 4.9 1.0X
RunLengthEncoding 1060 1061 2 63.3 15.8 0.3X
BooleanBitSet 1696 1717 29 39.6 25.3 0.2X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
SHORT Encode (Lower Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough(1.000) 9 10 1 7396.4 0.1 1.0X
RunLengthEncoding(1.495) 1412 1412 0 47.5 21.0 0.0X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
SHORT Decode (Lower Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough 1565 1588 32 42.9 23.3 1.0X
RunLengthEncoding 1841 1861 28 36.5 27.4 0.9X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
SHORT Encode (Higher Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough(1.000) 8 9 1 8316.3 0.1 1.0X
RunLengthEncoding(2.014) 1481 1489 10 45.3 22.1 0.0X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
SHORT Decode (Higher Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough 1537 1538 1 43.7 22.9 1.0X
RunLengthEncoding 1864 1870 9 36.0 27.8 0.8X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
INT Encode (Lower Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough(1.000) 17 19 1 3926.8 0.3 1.0X
RunLengthEncoding(1.002) 1483 1487 5 45.3 22.1 0.0X
DictionaryEncoding(0.500) 1158 1165 10 58.0 17.2 0.0X
IntDelta(0.250) 299 305 6 224.3 4.5 0.1X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
INT Decode (Lower Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough 1333 1343 14 50.3 19.9 1.0X
RunLengthEncoding 1944 1945 2 34.5 29.0 0.7X
DictionaryEncoding 925 945 18 72.6 13.8 1.4X
IntDelta 784 794 10 85.6 11.7 1.7X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
INT Encode (Higher Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough(1.000) 17 18 1 4021.0 0.2 1.0X
RunLengthEncoding(1.338) 1554 1576 32 43.2 23.1 0.0X
DictionaryEncoding(0.501) 1432 1438 8 46.9 21.3 0.0X
IntDelta(0.250) 303 314 10 221.7 4.5 0.1X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
INT Decode (Higher Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough 1414 1434 28 47.5 21.1 1.0X
RunLengthEncoding 2067 2077 14 32.5 30.8 0.7X
DictionaryEncoding 1187 1190 3 56.5 17.7 1.2X
IntDelta 1036 1072 51 64.8 15.4 1.4X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
LONG Encode (Lower Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough(1.000) 379 383 6 177.3 5.6 1.0X
RunLengthEncoding(0.754) 1618 1623 7 41.5 24.1 0.2X
DictionaryEncoding(0.250) 1246 1283 53 53.9 18.6 0.3X
LongDelta(0.125) 197 200 3 340.2 2.9 1.9X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
LONG Decode (Lower Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough 1567 1577 13 42.8 23.4 1.0X
RunLengthEncoding 2064 2080 22 32.5 30.8 0.8X
DictionaryEncoding 1252 1259 10 53.6 18.6 1.3X
LongDelta 922 930 8 72.8 13.7 1.7X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
LONG Encode (Higher Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough(1.000) 385 397 8 174.4 5.7 1.0X
RunLengthEncoding(0.998) 1687 1689 3 39.8 25.1 0.2X
DictionaryEncoding(0.251) 1453 1462 13 46.2 21.7 0.3X
LongDelta(0.125) 190 200 7 353.5 2.8 2.0X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
LONG Decode (Higher Skew): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough 1561 1564 3 43.0 23.3 1.0X
RunLengthEncoding 2066 2075 12 32.5 30.8 0.8X
DictionaryEncoding 1244 1252 11 53.9 18.5 1.3X
LongDelta 1114 1117 4 60.3 16.6 1.4X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
STRING Encode: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough(1.000) 584 592 8 114.9 8.7 1.0X
RunLengthEncoding(0.889) 2862 2872 14 23.4 42.7 0.2X
DictionaryEncoding(0.167) 3034 3053 26 22.1 45.2 0.2X
OpenJDK 64-Bit Server VM 11.0.10+9-LTS on Linux 5.4.0-1043-azure
Intel(R) Xeon(R) Platinum 8171M CPU @ 2.60GHz
STRING Decode: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
PassThrough 2937 2939 3 22.8 43.8 1.0X
RunLengthEncoding 3557 3574 24 18.9 53.0 0.8X
DictionaryEncoding 2858 2890 46 23.5 42.6 1.0X
Reference in a new issue View git blame Copy permalink