2018-10-04 14:58:16 -04:00
|
|
|
================================================================================================
|
|
|
|
Dataset Benchmark
|
|
|
|
================================================================================================
|
|
|
|
|
2021-04-03 16:02:56 -04:00
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
|
|
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
|
2019-10-03 11:58:25 -04:00
|
|
|
back-to-back map long: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
|
|
------------------------------------------------------------------------------------------------------------------------
|
2021-04-03 16:02:56 -04:00
|
|
|
RDD 12276 12399 175 8.1 122.8 1.0X
|
|
|
|
DataFrame 2017 2094 110 49.6 20.2 6.1X
|
|
|
|
Dataset 3034 3044 15 33.0 30.3 4.0X
|
2018-10-04 14:58:16 -04:00
|
|
|
|
2021-04-03 16:02:56 -04:00
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
|
|
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
|
2019-10-03 11:58:25 -04:00
|
|
|
back-to-back map: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
|
|
------------------------------------------------------------------------------------------------------------------------
|
2021-04-03 16:02:56 -04:00
|
|
|
RDD 16325 16361 50 6.1 163.2 1.0X
|
|
|
|
DataFrame 8463 8468 6 11.8 84.6 1.9X
|
|
|
|
Dataset 22525 23091 801 4.4 225.2 0.7X
|
2018-10-04 14:58:16 -04:00
|
|
|
|
2021-04-03 16:02:56 -04:00
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
|
|
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
|
2019-10-03 11:58:25 -04:00
|
|
|
back-to-back filter Long: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
|
|
------------------------------------------------------------------------------------------------------------------------
|
2021-04-03 16:02:56 -04:00
|
|
|
RDD 3133 3136 3 31.9 31.3 1.0X
|
|
|
|
DataFrame 1194 1535 482 83.8 11.9 2.6X
|
|
|
|
Dataset 3146 3156 13 31.8 31.5 1.0X
|
2018-10-04 14:58:16 -04:00
|
|
|
|
2021-04-03 16:02:56 -04:00
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
|
|
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
|
2019-10-03 11:58:25 -04:00
|
|
|
back-to-back filter: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
|
|
------------------------------------------------------------------------------------------------------------------------
|
2021-04-03 16:02:56 -04:00
|
|
|
RDD 5334 5352 25 18.7 53.3 1.0X
|
|
|
|
DataFrame 190 221 21 527.1 1.9 28.1X
|
|
|
|
Dataset 10536 10630 133 9.5 105.4 0.5X
|
2018-10-04 14:58:16 -04:00
|
|
|
|
2021-04-03 16:02:56 -04:00
|
|
|
OpenJDK 64-Bit Server VM 1.8.0_282-b08 on Linux 5.4.0-1043-azure
|
|
|
|
Intel(R) Xeon(R) CPU E5-2673 v3 @ 2.40GHz
|
2019-10-03 11:58:25 -04:00
|
|
|
aggregate: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
|
|
|
|
------------------------------------------------------------------------------------------------------------------------
|
2021-04-03 16:02:56 -04:00
|
|
|
RDD sum 4970 5099 181 20.1 49.7 1.0X
|
|
|
|
DataFrame sum 67 81 9 1483.8 0.7 73.8X
|
|
|
|
Dataset sum using Aggregator 9474 9771 420 10.6 94.7 0.5X
|
|
|
|
Dataset complex Aggregator 13975 14701 1028 7.2 139.7 0.4X
|
2018-10-04 14:58:16 -04:00
|
|
|
|
|
|
|
|