ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-36814][SQL] Make class ColumnarBatch extendable

Browse source 
### What changes were proposed in this pull request?
Change class ColumnarBatch to a non-final class

### Why are the changes needed?
To support better vectorized reading in multiple data source, ColumnarBatch need to be extendable. For example, To support row-level delete(  https://github.com/apache/iceberg/issues/3141) in Iceberg's vectorized read, we need to filter out deleted rows in a batch, which requires ColumnarBatch to be extendable.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
No test needed.

Closes #34054 from flyrain/columnarbatch-extendable.

Authored-by: Yufei Gu <yufei_gu@apple.com>
Signed-off-by: DB Tsai <d_tsai@apple.com>
This commit is contained in:
Yufei Gu 2021-09-21 15:24:55 +00:00 committed by DB Tsai
parent d2340f8e1c
commit 688b95b136
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java
View file

@ -31,7 +31,7 @@ import org.apache.spark.unsafe.types.UTF8String;
* the entire data loading process.
*/
@Evolving
public final class ColumnarBatch implements AutoCloseable {
public class ColumnarBatch implements AutoCloseable {
private int numRows;
private final ColumnVector[] columns;