3bba8621cf
## What changes were proposed in this pull request? This PR eliminates redundant null check in generated code for extracting an element from complex types `GetArrayItem`, `GetMapValue`, and `GetArrayStructFields`. Since these code generation does not take care of `nullable` in `DataType` such as `ArrayType`, the generated code always has `isNullAt(index)`. This PR avoids to generate `isNullAt(index)` if `nullable` is false in `DataType`. Example ``` val nonNullArray = Literal.create(Seq(1), ArrayType(IntegerType, false)) checkEvaluation(GetArrayItem(nonNullArray, Literal(0)), 1) ``` Before this PR ``` /* 034 */ public java.lang.Object apply(java.lang.Object _i) { /* 035 */ InternalRow i = (InternalRow) _i; /* 036 */ /* 037 */ /* 038 */ /* 039 */ boolean isNull = true; /* 040 */ int value = -1; /* 041 */ /* 042 */ /* 043 */ /* 044 */ isNull = false; // resultCode could change nullability. /* 045 */ /* 046 */ final int index = (int) 0; /* 047 */ if (index >= ((ArrayData) references[0]).numElements() || index < 0 || ((ArrayData) references[0]).isNullAt(index)) { /* 048 */ isNull = true; /* 049 */ } else { /* 050 */ value = ((ArrayData) references[0]).getInt(index); /* 051 */ } /* 052 */ isNull_0 = isNull; /* 053 */ value_0 = value; /* 054 */ /* 055 */ // copy all the results into MutableRow /* 056 */ /* 057 */ if (!isNull_0) { /* 058 */ mutableRow.setInt(0, value_0); /* 059 */ } else { /* 060 */ mutableRow.setNullAt(0); /* 061 */ } /* 062 */ /* 063 */ return mutableRow; /* 064 */ } ``` After this PR (Line 47 is changed) ``` /* 034 */ public java.lang.Object apply(java.lang.Object _i) { /* 035 */ InternalRow i = (InternalRow) _i; /* 036 */ /* 037 */ /* 038 */ /* 039 */ boolean isNull = true; /* 040 */ int value = -1; /* 041 */ /* 042 */ /* 043 */ /* 044 */ isNull = false; // resultCode could change nullability. /* 045 */ /* 046 */ final int index = (int) 0; /* 047 */ if (index >= ((ArrayData) references[0]).numElements() || index < 0) { /* 048 */ isNull = true; /* 049 */ } else { /* 050 */ value = ((ArrayData) references[0]).getInt(index); /* 051 */ } /* 052 */ isNull_0 = isNull; /* 053 */ value_0 = value; /* 054 */ /* 055 */ // copy all the results into MutableRow /* 056 */ /* 057 */ if (!isNull_0) { /* 058 */ mutableRow.setInt(0, value_0); /* 059 */ } else { /* 060 */ mutableRow.setNullAt(0); /* 061 */ } /* 062 */ /* 063 */ return mutableRow; /* 064 */ } ``` ## How was this patch tested? Added test cases into `ComplexTypeSuite` Author: Kazuaki Ishizaki <ishizaki@jp.ibm.com> Closes #19598 from kiszk/SPARK-22378. |
||
---|---|---|
.. | ||
src | ||
pom.xml |