49c062b2e0
## What changes were proposed in this pull request? Refactor ExternalAppendOnlyUnsafeRowArrayBenchmark to use main method. ## How was this patch tested? Manually tested and regenerated results. Please note that `spark.memory.debugFill` setting has a huge impact on this benchmark. Since it is set to true by default when running the benchmark from SBT, we need to disable it: ``` SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt ";project sql;set javaOptions in Test += \"-Dspark.memory.debugFill=false\";test:runMain org.apache.spark.sql.execution.ExternalAppendOnlyUnsafeRowArrayBenchmark" ``` Closes #22617 from peter-toth/SPARK-25484. Lead-authored-by: Peter Toth <peter.toth@gmail.com> Co-authored-by: Peter Toth <ptoth@hortonworks.com> Co-authored-by: Dongjoon Hyun <dongjoon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
46 lines
2.9 KiB
Plaintext
46 lines
2.9 KiB
Plaintext
================================================================================================
|
|
WITHOUT SPILL
|
|
================================================================================================
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
|
|
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
|
|
Array with 100000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------
|
|
ArrayBuffer 6378 / 6550 16.1 62.3 1.0X
|
|
ExternalAppendOnlyUnsafeRowArray 6196 / 6242 16.5 60.5 1.0X
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
|
|
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
|
|
Array with 1000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------
|
|
ArrayBuffer 11988 / 12027 21.9 45.7 1.0X
|
|
ExternalAppendOnlyUnsafeRowArray 37480 / 37574 7.0 143.0 0.3X
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
|
|
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
|
|
Array with 30000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------
|
|
ArrayBuffer 23536 / 23538 20.9 47.9 1.0X
|
|
ExternalAppendOnlyUnsafeRowArray 31275 / 31277 15.7 63.6 0.8X
|
|
|
|
|
|
================================================================================================
|
|
WITH SPILL
|
|
================================================================================================
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
|
|
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
|
|
Spilling with 1000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------
|
|
UnsafeExternalSorter 29241 / 29279 9.0 111.5 1.0X
|
|
ExternalAppendOnlyUnsafeRowArray 14309 / 14313 18.3 54.6 2.0X
|
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_191-b12 on Linux 3.10.0-862.3.2.el7.x86_64
|
|
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
|
|
Spilling with 10000 rows: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------
|
|
UnsafeExternalSorter 11 / 11 14.8 67.4 1.0X
|
|
ExternalAppendOnlyUnsafeRowArray 9 / 9 17.6 56.8 1.2X
|
|
|
|
|