[SPARK-34727][SQL] Fix discrepancy in casting float to timestamp
### What changes were proposed in this pull request? In non-ANSI mode, casting float to timestamp has different implementation for codegen on and off. Codegen on: 1. Multiply float input by MICROS_PER_SECOND 2. Cast resulting float value to long Codegen off: 1. CAST float input to double input 2. Multiply double input by MICROS_PER_SECOND 3. Cast resulting double value to long In the PR, I propose to align to non-codegen code, and cast input float to double in codegen. ### Why are the changes needed? This fixes the issue which is demonstrated by the code: ```sql spark-sql> CREATE TEMP VIEW v1 AS SELECT 16777215.0f AS f; spark-sql> SELECT * FROM v1; 1.6777215E7 spark-sql> SELECT CAST(f AS TIMESTAMP) FROM v1; 1970-07-14 07:20:15 spark-sql> CACHE TABLE v1; spark-sql> SELECT * FROM v1; 1.6777215E7 spark-sql> SELECT CAST(f AS TIMESTAMP) FROM v1; 1970-07-14 07:20:14.951424 ``` The result from the cached view **1970-07-14 07:20:14.951424** is different from un-cached view **1970-07-14 07:20:15**. ### Does this PR introduce _any_ user-facing change? Yes. After the changes, the example above outputs the same timestamp for the cached view: ```sql spark-sql> CACHE TABLE v1; spark-sql> SELECT * FROM v1; 1.6777215E7 spark-sql> SELECT CAST(f AS TIMESTAMP) FROM v1; 1970-07-14 07:20:15 ``` ### How was this patch tested? By running new test: ``` $ build/sbt "test:testOnly *CastSuite" ``` Closes #31819 from MaxGekk/fix-float-to-timestamp. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
parent
124b5af114
commit
e0a1399bd7
|
@ -1322,7 +1322,7 @@ abstract class CastBase extends UnaryExpression with TimeZoneAwareExpression wit
|
|||
if (Float.isNaN($c) || Float.isInfinite($c)) {
|
||||
$evNull = true;
|
||||
} else {
|
||||
$evPrim = (long)($c * $MICROS_PER_SECOND);
|
||||
$evPrim = (long)((double)$c * $MICROS_PER_SECOND);
|
||||
}
|
||||
"""
|
||||
}
|
||||
|
|
|
@ -1697,6 +1697,10 @@ class CastSuite extends CastSuiteBase {
|
|||
test("Cast from double II") {
|
||||
checkEvaluation(cast(cast(1.toDouble, TimestampType), DoubleType), 1.toDouble)
|
||||
}
|
||||
|
||||
test("SPARK-34727: cast from float II") {
|
||||
checkCast(16777215.0f, java.time.Instant.ofEpochSecond(16777215))
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
Loading…
Reference in a new issue