ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/sql/core/benchmarks/ExtractBenchmark-jdk11-results.txt
Dongjoon Hyun 854a0f752e [SPARK-29320][TESTS] Compare sql/core module in JDK8/11 (Part 1) 
### What changes were proposed in this pull request?

This PR regenerates the `sql/core` benchmarks in JDK8/11 to compare the result. In general, we compare the ratio instead of the time. However, in this PR, the average time is compared. This PR should be considered as a rough comparison.

**A. EXPECTED CASES(JDK11 is faster in general)**
- [x] BloomFilterBenchmark (JDK11 is faster except one case)
- [x] BuiltInDataSourceWriteBenchmark (JDK11 is faster at CSV/ORC)
- [x] CSVBenchmark (JDK11 is faster except five cases)
- [x] ColumnarBatchBenchmark (JDK11 is faster at `boolean`/`string` and some cases in `int`/`array`)
- [x] DatasetBenchmark (JDK11 is faster with `string`, but is slower for `long` type)
- [x] ExternalAppendOnlyUnsafeRowArrayBenchmark (JDK11 is faster except two cases)
- [x] ExtractBenchmark (JDK11 is faster except HOUR/MINUTE/SECOND/MILLISECONDS/MICROSECONDS)
- [x] HashedRelationMetricsBenchmark (JDK11 is faster)
- [x] JSONBenchmark (JDK11 is much faster except eight cases)
- [x] JoinBenchmark (JDK11 is faster except five cases)
- [x] OrcNestedSchemaPruningBenchmark (JDK11 is faster in nine cases)
- [x] PrimitiveArrayBenchmark (JDK11 is faster)
- [x] SortBenchmark (JDK11 is faster except `Arrays.sort` case)
- [x] UDFBenchmark (N/A, values are too small)
- [x] UnsafeArrayDataBenchmark (JDK11 is faster except one case)
- [x] WideTableBenchmark (JDK11 is faster except two cases)

**B. CASES WE NEED TO INVESTIGATE MORE LATER**
- [x] AggregateBenchmark (JDK11 is slower in general)
- [x] CompressionSchemeBenchmark (JDK11 is slower in general except `string`)
- [x] DataSourceReadBenchmark (JDK11 is slower in general)
- [x] DateTimeBenchmark (JDK11 is slightly slower in general except `parsing`)
- [x] MakeDateTimeBenchmark (JDK11 is slower except two cases)
- [x] MiscBenchmark (JDK11 is slower except ten cases)
- [x] OrcV2NestedSchemaPruningBenchmark (JDK11 is slower)
- [x] ParquetNestedSchemaPruningBenchmark (JDK11 is slower except six cases)
- [x] RangeBenchmark (JDK11 is slower except one case)

`FilterPushdownBenchmark/InExpressionBenchmark/WideSchemaBenchmark` will be compared later because it took long timer.

### Why are the changes needed?

According to the result, there are some difference between JDK8/JDK11.
This will be a baseline for the future improvement and comparison. Also, as a reproducible  environment, the following environment is used.
- Instance: `r3.xlarge`
- OS: `CentOS Linux release 7.5.1804 (Core)`
- JDK:
  - `OpenJDK Runtime Environment (build 1.8.0_222-b10)`
  - `OpenJDK Runtime Environment 18.9 (build 11.0.4+11-LTS)`

### Does this PR introduce any user-facing change?

No.

### How was this patch tested?

This is a test-only PR. We need to run benchmark.

Closes #26003 from dongjoon-hyun/SPARK-29320.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2019-10-03 08:58:25 -07:00

101 lines
11 KiB
Plaintext
Raw Blame History

OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Invoke extract for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp 342 456 120 29.2 34.2 1.0X
MILLENNIUM of timestamp 1462 1498 53 6.8 146.2 0.2X
CENTURY of timestamp 1229 1293 86 8.1 122.9 0.3X
DECADE of timestamp 1204 1242 56 8.3 120.4 0.3X
YEAR of timestamp 1180 1226 72 8.5 118.0 0.3X
ISOYEAR of timestamp 1279 1301 33 7.8 127.9 0.3X
QUARTER of timestamp 1357 1379 22 7.4 135.7 0.3X
MONTH of timestamp 1155 1182 24 8.7 115.5 0.3X
WEEK of timestamp 1634 1641 10 6.1 163.4 0.2X
DAY of timestamp 1167 1171 6 8.6 116.7 0.3X
DAYOFWEEK of timestamp 1335 1344 8 7.5 133.5 0.3X
DOW of timestamp 1344 1352 11 7.4 134.4 0.3X
ISODOW of timestamp 1279 1281 2 7.8 127.9 0.3X
DOY of timestamp 1172 1183 16 8.5 117.2 0.3X
HOUR of timestamp 722 731 12 13.8 72.2 0.5X
MINUTE of timestamp 725 734 9 13.8 72.5 0.5X
SECOND of timestamp 656 661 6 15.3 65.6 0.5X
MILLISECONDS of timestamp 630 637 9 15.9 63.0 0.5X
MICROSECONDS of timestamp 587 592 7 17.0 58.7 0.6X
EPOCH of timestamp 1036 1047 13 9.7 103.6 0.3X
OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Invoke extract for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to date 1044 1081 63 9.6 104.4 1.0X
MILLENNIUM of date 1367 1375 10 7.3 136.7 0.8X
CENTURY of date 1181 1183 3 8.5 118.1 0.9X
DECADE of date 1154 1163 8 8.7 115.4 0.9X
YEAR of date 1138 1154 16 8.8 113.8 0.9X
ISOYEAR of date 1394 1401 8 7.2 139.4 0.7X
QUARTER of date 1336 1362 26 7.5 133.6 0.8X
MONTH of date 1137 1144 11 8.8 113.7 0.9X
WEEK of date 1619 1632 18 6.2 161.9 0.6X
DAY of date 1144 1158 22 8.7 114.4 0.9X
DAYOFWEEK of date 1328 1330 2 7.5 132.8 0.8X
DOW of date 1336 1339 4 7.5 133.6 0.8X
ISODOW of date 1272 1276 3 7.9 127.2 0.8X
DOY of date 1170 1174 4 8.6 117.0 0.9X
HOUR of date 1963 1986 35 5.1 196.3 0.5X
MINUTE of date 1969 1984 19 5.1 196.9 0.5X
SECOND of date 1926 1929 3 5.2 192.6 0.5X
MILLISECONDS of date 1928 1939 16 5.2 192.8 0.5X
MICROSECONDS of date 1816 1821 4 5.5 181.6 0.6X
EPOCH of date 2331 2336 4 4.3 233.1 0.4X
OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Invoke date_part for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp 315 320 8 31.8 31.5 1.0X
MILLENNIUM of timestamp 1365 1369 4 7.3 136.5 0.2X
CENTURY of timestamp 1176 1179 4 8.5 117.6 0.3X
DECADE of timestamp 1153 1168 14 8.7 115.3 0.3X
YEAR of timestamp 1131 1135 5 8.8 113.1 0.3X
ISOYEAR of timestamp 1388 1397 8 7.2 138.8 0.2X
QUARTER of timestamp 1344 1363 25 7.4 134.4 0.2X
MONTH of timestamp 1135 1141 6 8.8 113.5 0.3X
WEEK of timestamp 1612 1615 5 6.2 161.2 0.2X
DAY of timestamp 1138 1145 7 8.8 113.8 0.3X
DAYOFWEEK of timestamp 1322 1330 7 7.6 132.2 0.2X
DOW of timestamp 1317 1323 5 7.6 131.7 0.2X
ISODOW of timestamp 1284 1287 4 7.8 128.4 0.2X
DOY of timestamp 1163 1172 11 8.6 116.3 0.3X
HOUR of timestamp 709 709 1 14.1 70.9 0.4X
MINUTE of timestamp 713 714 1 14.0 71.3 0.4X
SECOND of timestamp 635 641 6 15.7 63.5 0.5X
MILLISECONDS of timestamp 625 629 7 16.0 62.5 0.5X
MICROSECONDS of timestamp 572 573 2 17.5 57.2 0.6X
EPOCH of timestamp 1031 1033 3 9.7 103.1 0.3X
OpenJDK 64-Bit Server VM 11.0.4+11-LTS on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Invoke date_part for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to date 994 998 3 10.1 99.4 1.0X
MILLENNIUM of date 1361 1371 9 7.3 136.1 0.7X
CENTURY of date 1171 1180 15 8.5 117.1 0.8X
DECADE of date 1149 1153 4 8.7 114.9 0.9X
YEAR of date 1128 1130 2 8.9 112.8 0.9X
ISOYEAR of date 1384 1398 19 7.2 138.4 0.7X
QUARTER of date 1333 1345 19 7.5 133.3 0.7X
MONTH of date 1132 1138 7 8.8 113.2 0.9X
WEEK of date 1616 1624 9 6.2 161.6 0.6X
DAY of date 1132 1135 3 8.8 113.2 0.9X
DAYOFWEEK of date 1320 1322 3 7.6 132.0 0.8X
DOW of date 1317 1319 2 7.6 131.7 0.8X
ISODOW of date 1269 1272 5 7.9 126.9 0.8X
DOY of date 1171 1190 17 8.5 117.1 0.8X
HOUR of date 1960 1962 2 5.1 196.0 0.5X
MINUTE of date 1974 1975 1 5.1 197.4 0.5X
SECOND of date 1915 1922 8 5.2 191.5 0.5X
MILLISECONDS of date 1923 1924 2 5.2 192.3 0.5X
MICROSECONDS of date 1817 1819 1 5.5 181.7 0.5X
EPOCH of date 2323 2326 2 4.3 232.3 0.4X
Reference in a new issue View git blame Copy permalink