ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[MINOR][SQL][FOLLOW-UP] Add assertion to FixedLengthRowBasedKeyValueBatch

Browse source 
### What changes were proposed in this pull request?
Adds an assert to `FixedLengthRowBasedKeyValueBatch#appendRow` method to check the incoming vlen and klen by comparing them with the lengths stored as member variables as followup to https://github.com/apache/spark/pull/30788

### Why are the changes needed?
Add assert statement to catch similar bugs in future.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
Ran some tests locally, though not easy to test.

Closes #31447 from yliou/SPARK-33726-Assert.

Authored-by: yliou <yliou@berkeley.edu>
Signed-off-by: Sean Owen <srowen@gmail.com>
This commit is contained in:
yliou 2021-02-08 08:46:01 -06:00 committed by Sean Owen
parent 037bfb2dbc
commit d1131bc850
1 changed files with 2 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
sql/catalyst/src/main/java/org/apache/spark/sql/catalyst/expressions/FixedLengthRowBasedKeyValueBatch.java
View file

@ -46,6 +46,8 @@ public final class FixedLengthRowBasedKeyValueBatch extends RowBasedKeyValueBatc
public UnsafeRow appendRow(Object kbase, long koff, int klen,
Object vbase, long voff, int vlen) {
// if run out of max supported rows or page size, return null
assert(vlen == this.vlen);
assert(klen == this.klen);
if (numRows >= capacity || page == null || page.size() - pageCursor < recordLength) {
return null;
}