ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/sql/core/benchmarks/ExtractBenchmark-results.txt
Maxim Gekk 89bad267d4 [SPARK-29200][SQL] Optimize extract/date_part for epoch 
### What changes were proposed in this pull request?

Refactoring of the `DateTimeUtils.getEpoch()` function by avoiding decimal operations that are pretty expensive, and converting the final result to the decimal type at the end.

### Why are the changes needed?
The changes improve performance of the `getEpoch()` method at least up to **20 times**.
Before:
```
Invoke extract for timestamp:             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp                                   256            277          33         39.0          25.6       1.0X
EPOCH of timestamp                                23455          23550         131          0.4        2345.5       0.0X
```
After:
```
Invoke extract for timestamp:             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp                                   255            294          34         39.2          25.5       1.0X
EPOCH of timestamp                                 1049           1054           9          9.5         104.9       0.2X
```

### Does this PR introduce any user-facing change?
No

### How was this patch tested?

By existing test from `DateExpressionSuite`.

Closes #25881 from MaxGekk/optimize-extract-epoch.

Authored-by: Maxim Gekk <max.gekk@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
2019-09-22 16:59:59 +09:00

101 lines
11 KiB
Plaintext
Raw Blame History

Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.6
Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
Invoke extract for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp 254 275 35 39.4 25.4 1.0X
MILLENNIUM of timestamp 1149 1159 9 8.7 114.9 0.2X
CENTURY of timestamp 1102 1115 16 9.1 110.2 0.2X
DECADE of timestamp 1024 1036 11 9.8 102.4 0.2X
YEAR of timestamp 1000 1004 5 10.0 100.0 0.3X
ISOYEAR of timestamp 1090 1101 11 9.2 109.0 0.2X
QUARTER of timestamp 1169 1178 7 8.6 116.9 0.2X
MONTH of timestamp 981 984 4 10.2 98.1 0.3X
WEEK of timestamp 1372 1388 14 7.3 137.2 0.2X
DAY of timestamp 994 1000 7 10.1 99.4 0.3X
DAYOFWEEK of timestamp 1102 1108 6 9.1 110.2 0.2X
DOW of timestamp 1102 1105 3 9.1 110.2 0.2X
ISODOW of timestamp 1063 1078 18 9.4 106.3 0.2X
DOY of timestamp 1015 1021 5 9.9 101.5 0.2X
HOUR of timestamp 385 390 5 26.0 38.5 0.7X
MINUTE of timestamp 387 391 7 25.8 38.7 0.7X
SECOND of timestamp 381 382 1 26.2 38.1 0.7X
MILLISECONDS of timestamp 584 588 4 17.1 58.4 0.4X
MICROSECONDS of timestamp 482 500 16 20.8 48.2 0.5X
EPOCH of timestamp 957 961 6 10.4 95.7 0.3X
Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.6
Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
Invoke extract for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to date 834 842 12 12.0 83.4 1.0X
MILLENNIUM of date 1078 1088 16 9.3 107.8 0.8X
CENTURY of date 1063 1067 4 9.4 106.3 0.8X
DECADE of date 989 992 3 10.1 98.9 0.8X
YEAR of date 975 976 1 10.3 97.5 0.9X
ISOYEAR of date 1177 1186 9 8.5 117.7 0.7X
QUARTER of date 1219 1222 4 8.2 121.9 0.7X
MONTH of date 982 992 10 10.2 98.2 0.8X
WEEK of date 1360 1364 6 7.4 136.0 0.6X
DAY of date 973 980 7 10.3 97.3 0.9X
DAYOFWEEK of date 1100 1104 7 9.1 110.0 0.8X
DOW of date 1091 1096 4 9.2 109.1 0.8X
ISODOW of date 1053 1057 6 9.5 105.3 0.8X
DOY of date 1006 1012 4 9.9 100.6 0.8X
HOUR of date 1683 1688 6 5.9 168.3 0.5X
MINUTE of date 1686 1691 5 5.9 168.6 0.5X
SECOND of date 1706 1714 7 5.9 170.6 0.5X
MILLISECONDS of date 1881 1887 6 5.3 188.1 0.4X
MICROSECONDS of date 1767 1778 16 5.7 176.7 0.5X
EPOCH of date 2274 2281 7 4.4 227.4 0.4X
Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.6
Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
Invoke date_part for timestamp: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to timestamp 212 214 3 47.2 21.2 1.0X
MILLENNIUM of timestamp 1082 1092 10 9.2 108.2 0.2X
CENTURY of timestamp 1087 1089 2 9.2 108.7 0.2X
DECADE of timestamp 986 1001 21 10.1 98.6 0.2X
YEAR of timestamp 983 985 2 10.2 98.3 0.2X
ISOYEAR of timestamp 1160 1168 12 8.6 116.0 0.2X
QUARTER of timestamp 1219 1228 9 8.2 121.9 0.2X
MONTH of timestamp 973 979 5 10.3 97.3 0.2X
WEEK of timestamp 1348 1357 8 7.4 134.8 0.2X
DAY of timestamp 969 974 5 10.3 96.9 0.2X
DAYOFWEEK of timestamp 1093 1096 3 9.1 109.3 0.2X
DOW of timestamp 1089 1096 7 9.2 108.9 0.2X
ISODOW of timestamp 1048 1056 7 9.5 104.8 0.2X
DOY of timestamp 1007 1011 5 9.9 100.7 0.2X
HOUR of timestamp 385 387 3 26.0 38.5 0.6X
MINUTE of timestamp 382 384 2 26.2 38.2 0.6X
SECOND of timestamp 377 378 1 26.5 37.7 0.6X
MILLISECONDS of timestamp 572 574 2 17.5 57.2 0.4X
MICROSECONDS of timestamp 473 490 18 21.1 47.3 0.4X
EPOCH of timestamp 937 947 11 10.7 93.7 0.2X
Java HotSpot(TM) 64-Bit Server VM 1.8.0_202-b08 on Mac OS X 10.14.6
Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
Invoke date_part for date: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
cast to date 835 836 1 12.0 83.5 1.0X
MILLENNIUM of date 1062 1070 10 9.4 106.2 0.8X
CENTURY of date 1060 1068 12 9.4 106.0 0.8X
DECADE of date 984 992 8 10.2 98.4 0.8X
YEAR of date 971 973 3 10.3 97.1 0.9X
ISOYEAR of date 1158 1175 22 8.6 115.8 0.7X
QUARTER of date 1239 1239 0 8.1 123.9 0.7X
MONTH of date 972 987 23 10.3 97.2 0.9X
WEEK of date 1346 1350 5 7.4 134.6 0.6X
DAY of date 970 971 1 10.3 97.0 0.9X
DAYOFWEEK of date 1099 1107 10 9.1 109.9 0.8X
DOW of date 1089 1091 2 9.2 108.9 0.8X
ISODOW of date 1053 1062 13 9.5 105.3 0.8X
DOY of date 1008 1010 4 9.9 100.8 0.8X
HOUR of date 1697 1703 7 5.9 169.7 0.5X
MINUTE of date 1697 1699 3 5.9 169.7 0.5X
SECOND of date 1682 1692 8 5.9 168.2 0.5X
MILLISECONDS of date 1884 1890 7 5.3 188.4 0.4X
MICROSECONDS of date 1765 1770 7 5.7 176.5 0.5X
EPOCH of date 2273 2282 8 4.4 227.3 0.4X
Reference in a new issue View git blame Copy permalink