[SPARK-36814][SQL] Make class ColumnarBatch extendable
### What changes were proposed in this pull request? Change class ColumnarBatch to a non-final class ### Why are the changes needed? To support better vectorized reading in multiple data source, ColumnarBatch need to be extendable. For example, To support row-level delete( https://github.com/apache/iceberg/issues/3141) in Iceberg's vectorized read, we need to filter out deleted rows in a batch, which requires ColumnarBatch to be extendable. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No test needed. Closes #34054 from flyrain/columnarbatch-extendable. Authored-by: Yufei Gu <yufei_gu@apple.com> Signed-off-by: DB Tsai <d_tsai@apple.com>
This commit is contained in:
parent
d2340f8e1c
commit
688b95b136
|
@ -31,7 +31,7 @@ import org.apache.spark.unsafe.types.UTF8String;
|
||||||
* the entire data loading process.
|
* the entire data loading process.
|
||||||
*/
|
*/
|
||||||
@Evolving
|
@Evolving
|
||||||
public final class ColumnarBatch implements AutoCloseable {
|
public class ColumnarBatch implements AutoCloseable {
|
||||||
private int numRows;
|
private int numRows;
|
||||||
private final ColumnVector[] columns;
|
private final ColumnVector[] columns;
|
||||||
|
|
||||||
|
|
Loading…
Reference in a new issue