ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/sql/core/benchmarks/ExtractBenchmark-results.txt
Dongjoon Hyun 854a0f752e [SPARK-29320][TESTS] Compare sql/core module in JDK8/11 (Part 1) 
### What changes were proposed in this pull request?

This PR regenerates the `sql/core` benchmarks in JDK8/11 to compare the result. In general, we compare the ratio instead of the time. However, in this PR, the average time is compared. This PR should be considered as a rough comparison.

**A. EXPECTED CASES(JDK11 is faster in general)**
- [x] BloomFilterBenchmark (JDK11 is faster except one case)
- [x] BuiltInDataSourceWriteBenchmark (JDK11 is faster at CSV/ORC)
- [x] CSVBenchmark (JDK11 is faster except five cases)
- [x] ColumnarBatchBenchmark (JDK11 is faster at `boolean`/`string` and some cases in `int`/`array`)
- [x] DatasetBenchmark (JDK11 is faster with `string`, but is slower for `long` type)
- [x] ExternalAppendOnlyUnsafeRowArrayBenchmark (JDK11 is faster except two cases)
- [x] ExtractBenchmark (JDK11 is faster except HOUR/MINUTE/SECOND/MILLISECONDS/MICROSECONDS)
- [x] HashedRelationMetricsBenchmark (JDK11 is faster)
- [x] JSONBenchmark (JDK11 is much faster except eight cases)
- [x] JoinBenchmark (JDK11 is faster except five cases)
- [x] OrcNestedSchemaPruningBenchmark (JDK11 is faster in nine cases)
- [x] PrimitiveArrayBenchmark (JDK11 is faster)
- [x] SortBenchmark (JDK11 is faster except `Arrays.sort` case)
- [x] UDFBenchmark (N/A, values are too small)
- [x] UnsafeArrayDataBenchmark (JDK11 is faster except one case)
- [x] WideTableBenchmark (JDK11 is faster except two cases)

**B. CASES WE NEED TO INVESTIGATE MORE LATER**
- [x] AggregateBenchmark (JDK11 is slower in general)
- [x] CompressionSchemeBenchmark (JDK11 is slower in general except `string`)
- [x] DataSourceReadBenchmark (JDK11 is slower in general)
- [x] DateTimeBenchmark (JDK11 is slightly slower in general except `parsing`)
- [x] MakeDateTimeBenchmark (JDK11 is slower except two cases)
- [x] MiscBenchmark (JDK11 is slower except ten cases)
- [x] OrcV2NestedSchemaPruningBenchmark (JDK11 is slower)
- [x] ParquetNestedSchemaPruningBenchmark (JDK11 is slower except six cases)
- [x] RangeBenchmark (JDK11 is slower except one case)

`FilterPushdownBenchmark/InExpressionBenchmark/WideSchemaBenchmark` will be compared later because it took long timer.

### Why are the changes needed?

According to the result, there are some difference between JDK8/JDK11.
This will be a baseline for the future improvement and comparison. Also, as a reproducible  environment, the following environment is used.
- Instance: `r3.xlarge`
- OS: `CentOS Linux release 7.5.1804 (Core)`
- JDK:
  - `OpenJDK Runtime Environment (build 1.8.0_222-b10)`
  - `OpenJDK Runtime Environment 18.9 (build 11.0.4+11-LTS)`

### Does this PR introduce any user-facing change?

No.

### How was this patch tested?

This is a test-only PR. We need to run benchmark.

Closes #26003 from dongjoon-hyun/SPARK-29320.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2019-10-03 08:58:25 -07:00

101 lines
11 KiB
Plaintext
Raw Blame History

OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Invoke extract for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp 404 460 71 24.8 40.4 1.0X
MILLENNIUM of timestamp 1432 1580 128 7.0 143.2 0.3X
CENTURY of timestamp 1380 1390 10 7.2 138.0 0.3X
DECADE of timestamp 1254 1261 8 8.0 125.4 0.3X
YEAR of timestamp 1229 1236 9 8.1 122.9 0.3X
ISOYEAR of timestamp 1382 1415 44 7.2 138.2 0.3X
QUARTER of timestamp 1444 1458 16 6.9 144.4 0.3X
MONTH of timestamp 1212 1257 60 8.2 121.2 0.3X
WEEK of timestamp 1718 1730 12 5.8 171.8 0.2X
DAY of timestamp 1180 1201 22 8.5 118.0 0.3X
DAYOFWEEK of timestamp 1371 1427 68 7.3 137.1 0.3X
DOW of timestamp 1365 1378 18 7.3 136.5 0.3X
ISODOW of timestamp 1312 1319 7 7.6 131.2 0.3X
DOY of timestamp 1224 1238 16 8.2 122.4 0.3X
HOUR of timestamp 343 355 16 29.2 34.3 1.2X
MINUTE of timestamp 341 344 3 29.3 34.1 1.2X
SECOND of timestamp 553 556 2 18.1 55.3 0.7X
MILLISECONDS of timestamp 562 565 3 17.8 56.2 0.7X
MICROSECONDS of timestamp 435 465 49 23.0 43.5 0.9X
EPOCH of timestamp 1211 1217 9 8.3 121.1 0.3X
OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Invoke extract for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to date 1030 1038 8 9.7 103.0 1.0X
MILLENNIUM of date 1302 1306 6 7.7 130.2 0.8X
CENTURY of date 1309 1315 6 7.6 130.9 0.8X
DECADE of date 1197 1202 7 8.4 119.7 0.9X
YEAR of date 1182 1192 10 8.5 118.2 0.9X
ISOYEAR of date 1421 1429 10 7.0 142.1 0.7X
QUARTER of date 1454 1468 14 6.9 145.4 0.7X
MONTH of date 1217 1230 19 8.2 121.7 0.8X
WEEK of date 1723 1725 3 5.8 172.3 0.6X
DAY of date 1188 1191 4 8.4 118.8 0.9X
DAYOFWEEK of date 1348 1355 6 7.4 134.8 0.8X
DOW of date 1344 1353 8 7.4 134.4 0.8X
ISODOW of date 1301 1311 9 7.7 130.1 0.8X
DOY of date 1237 1241 4 8.1 123.7 0.8X
HOUR of date 1465 1474 16 6.8 146.5 0.7X
MINUTE of date 1463 1471 8 6.8 146.3 0.7X
SECOND of date 1720 1725 5 5.8 172.0 0.6X
MILLISECONDS of date 1739 1749 9 5.8 173.9 0.6X
MICROSECONDS of date 1607 1611 4 6.2 160.7 0.6X
EPOCH of date 2370 2377 10 4.2 237.0 0.4X
OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Invoke date_part for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp 307 312 6 32.6 30.7 1.0X
MILLENNIUM of timestamp 1294 1298 4 7.7 129.4 0.2X
CENTURY of timestamp 1317 1321 6 7.6 131.7 0.2X
DECADE of timestamp 1202 1207 8 8.3 120.2 0.3X
YEAR of timestamp 1196 1201 8 8.4 119.6 0.3X
ISOYEAR of timestamp 1431 1440 10 7.0 143.1 0.2X
QUARTER of timestamp 1483 1487 5 6.7 148.3 0.2X
MONTH of timestamp 1205 1209 4 8.3 120.5 0.3X
WEEK of timestamp 1728 1731 4 5.8 172.8 0.2X
DAY of timestamp 1190 1193 3 8.4 119.0 0.3X
DAYOFWEEK of timestamp 1369 1371 2 7.3 136.9 0.2X
DOW of timestamp 1371 1373 3 7.3 137.1 0.2X
ISODOW of timestamp 1316 1321 8 7.6 131.6 0.2X
DOY of timestamp 1236 1241 6 8.1 123.6 0.2X
HOUR of timestamp 342 343 1 29.3 34.2 0.9X
MINUTE of timestamp 338 352 23 29.6 33.8 0.9X
SECOND of timestamp 555 556 1 18.0 55.5 0.6X
MILLISECONDS of timestamp 570 573 4 17.6 57.0 0.5X
MICROSECONDS of timestamp 436 442 5 22.9 43.6 0.7X
EPOCH of timestamp 1212 1218 7 8.3 121.2 0.3X
OpenJDK 64-Bit Server VM 1.8.0_222-b10 on Linux 3.10.0-862.3.2.el7.x86_64
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Invoke date_part for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to date 1017 1025 13 9.8 101.7 1.0X
MILLENNIUM of date 1314 1318 5 7.6 131.4 0.8X
CENTURY of date 1318 1334 16 7.6 131.8 0.8X
DECADE of date 1205 1210 5 8.3 120.5 0.8X
YEAR of date 1187 1190 3 8.4 118.7 0.9X
ISOYEAR of date 1409 1432 21 7.1 140.9 0.7X
QUARTER of date 1477 1482 6 6.8 147.7 0.7X
MONTH of date 1193 1199 7 8.4 119.3 0.9X
WEEK of date 1711 1719 7 5.8 171.1 0.6X
DAY of date 1183 1192 7 8.5 118.3 0.9X
DAYOFWEEK of date 1368 1371 6 7.3 136.8 0.7X
DOW of date 1361 1366 8 7.3 136.1 0.7X
ISODOW of date 1312 1315 3 7.6 131.2 0.8X
DOY of date 1230 1233 2 8.1 123.0 0.8X
HOUR of date 1480 1483 2 6.8 148.0 0.7X
MINUTE of date 1473 1489 27 6.8 147.3 0.7X
SECOND of date 1731 1737 7 5.8 173.1 0.6X
MILLISECONDS of date 1744 1749 6 5.7 174.4 0.6X
MICROSECONDS of date 1592 1594 1 6.3 159.2 0.6X
EPOCH of date 2368 2371 3 4.2 236.8 0.4X
Reference in a new issue View git blame Copy permalink