[SPARK-29387][SQL][FOLLOWUP] Fix issues of the multiply and divide for intervals

### What changes were proposed in this pull request?

Handle the inconsistence dividing zeros between literals and columns.
fix the null issue too.

### Why are the changes needed?
BUG FIX
### 1 Handle the inconsistence dividing zeros between literals and columns
```sql
-- !query 24
select
    k,
    v,
    cast(k as interval) / v,
    cast(k as interval) * v
from VALUES
     ('1 seconds', 1),
     ('2 seconds', 0),
     ('3 seconds', null),
     (null, null),
     (null, 0) t(k, v)
-- !query 24 schema
struct<k:string,v:int,divide_interval(CAST(k AS INTERVAL), CAST(v AS DOUBLE)):interval,multiply_interval(CAST(k AS INTERVAL), CAST(v AS DOUBLE)):interval>
-- !query 24 output
1 seconds   1   interval 1 seconds  interval 1 seconds
2 seconds   0   interval 0 microseconds interval 0 microseconds
3 seconds   NULL    NULL    NULL
NULL    0   NULL    NULL
NULL    NULL    NULL    NULL
```
```sql
-- !query 21
select interval '1 year 2 month' / 0
-- !query 21 schema
struct<divide_interval(interval 1 years 2 months, CAST(0 AS DOUBLE)):interval>
-- !query 21 output
NULL
```

in the first case, interval ’2 seconds ‘ / 0, it produces `interval 0 microseconds `
in the second case, it is `null`

### 2 null literal issues

```sql

  -- !query 20
select interval '1 year 2 month' / null
-- !query 20 schema
struct<>
-- !query 20 output
org.apache.spark.sql.AnalysisException
cannot resolve '(interval 1 years 2 months / NULL)' due to data type mismatch: differing types in '(interval 1 years 2 months / NULL)' (interval and null).; line 1 pos 7

-- !query 22
select interval '4 months 2 weeks 6 days' * null
-- !query 22 schema
struct<>
-- !query 22 output
org.apache.spark.sql.AnalysisException
cannot resolve '(interval 4 months 20 days * NULL)' due to data type mismatch: differing types in '(interval 4 months 20 days * NULL)' (interval and null).; line 1 pos 7

-- !query 23
select null * interval '4 months 2 weeks 6 days'
-- !query 23 schema
struct<>
-- !query 23 output
org.apache.spark.sql.AnalysisException
cannot resolve '(NULL * interval 4 months 20 days)' due to data type mismatch: differing types in '(NULL * interval 4 months 20 days)' (null and interval).; line 1 pos 7
```
 dividing or multiplying null literals, error occurs; where in column is fine as the first case
### Does this PR introduce any user-facing change?

NO, maybe yes, but it is just a follow-up

### How was this patch tested?

add uts

cc cloud-fan MaxGekk maropu

Closes #26410 from yaooqinn/SPARK-29387.

Authored-by: Kent Yao <yaooqinn@hotmail.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
This commit is contained in:
Kent Yao 2019-11-07 12:19:03 +08:00 committed by Wenchen Fan
parent 252ecd333f
commit 3437862975
6 changed files with 47 additions and 4 deletions

View file

@ -879,8 +879,12 @@ object TypeCoercion {
case e if !e.childrenResolved => e
// If DecimalType operands are involved, DecimalPrecision will handle it
// If CalendarIntervalType operands are involved, DateTimeOperations will handle it
case b @ BinaryOperator(left, right) if !left.dataType.isInstanceOf[DecimalType] &&
!right.dataType.isInstanceOf[DecimalType] && left.dataType != right.dataType =>
!right.dataType.isInstanceOf[DecimalType] &&
!left.dataType.isInstanceOf[CalendarIntervalType] &&
!right.dataType.isInstanceOf[CalendarIntervalType] &&
left.dataType != right.dataType =>
findTightestCommonType(left.dataType, right.dataType).map { commonType =>
if (b.inputType.acceptsType(commonType)) {
// If the expression accepts the tightest common type, cast to that.

View file

@ -386,6 +386,7 @@ object IntervalUtils {
}
def divide(interval: CalendarInterval, num: Double): CalendarInterval = {
if (num == 0) throw new java.lang.ArithmeticException("divide by zero")
fromDoubles(interval.months / num, interval.days / num, interval.microseconds / num)
}
}

View file

@ -224,6 +224,6 @@ class IntervalExpressionsSuite extends SparkFunSuite with ExpressionEvalHelper {
check("2 years -8 seconds", 0.5, "4 years -16 seconds")
check("-1 month 2 microseconds", -0.25, "4 months -8 microseconds")
check("1 month 3 microsecond", 1.5, "20 days 2 microseconds")
check("2 months", 0, null)
check("1 second", 0, null)
}
}

View file

@ -222,7 +222,7 @@ class IntervalUtilsSuite extends SparkFunSuite {
fail("Expected to throw an exception on divide by zero")
} catch {
case e: ArithmeticException =>
assert(e.getMessage.contains("overflow"))
assert(e.getMessage.contains("divide by zero"))
}
}
}

View file

@ -41,3 +41,9 @@ select timestamp'2019-10-06 10:11:12.345678' - date'2020-01-01';
select 3 * (timestamp'2019-10-15 10:11:12.001002' - date'2019-10-15');
select interval 4 month 2 weeks 3 microseconds * 1.5;
select (timestamp'2019-10-15' - timestamp'2019-10-14') / 1.5;
-- interval operation with null and zero case
select interval '2 seconds' / 0;
select interval '2 seconds' / null;
select interval '2 seconds' * null;
select null * interval '2 seconds';

View file

@ -1,5 +1,5 @@
-- Automatically generated by SQLQueryTestSuite
-- Number of queries: 20
-- Number of queries: 24
-- !query 0
@ -169,3 +169,35 @@ select (timestamp'2019-10-15' - timestamp'2019-10-14') / 1.5
struct<divide_interval(subtracttimestamps(TIMESTAMP('2019-10-15 00:00:00'), TIMESTAMP('2019-10-14 00:00:00')), CAST(1.5 AS DOUBLE)):interval>
-- !query 19 output
interval 16 hours
-- !query 20
select interval '2 seconds' / 0
-- !query 20 schema
struct<divide_interval(interval 2 seconds, CAST(0 AS DOUBLE)):interval>
-- !query 20 output
NULL
-- !query 21
select interval '2 seconds' / null
-- !query 21 schema
struct<divide_interval(interval 2 seconds, CAST(NULL AS DOUBLE)):interval>
-- !query 21 output
NULL
-- !query 22
select interval '2 seconds' * null
-- !query 22 schema
struct<multiply_interval(interval 2 seconds, CAST(NULL AS DOUBLE)):interval>
-- !query 22 output
NULL
-- !query 23
select null * interval '2 seconds'
-- !query 23 schema
struct<multiply_interval(interval 2 seconds, CAST(NULL AS DOUBLE)):interval>
-- !query 23 output
NULL