spark-instrumented-optimizer/sql/core
Wenchen Fan 71c24aad36 [SPARK-25602][SQL] SparkPlan.getByteArrayRdd should not consume the input when not necessary
## What changes were proposed in this pull request?

In `SparkPlan.getByteArrayRdd`, we should only call `it.hasNext` when the limit is not hit, as `iter.hasNext` may produce one row and buffer it, and cause wrong metrics.

## How was this patch tested?

new tests

Closes #22621 from cloud-fan/range.

Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2018-10-04 20:15:21 +08:00
..
benchmarks [SPARK-25483][TEST] Refactor UnsafeArrayDataBenchmark to use main method 2018-10-03 04:20:02 -07:00
src [SPARK-25602][SQL] SparkPlan.getByteArrayRdd should not consume the input when not necessary 2018-10-04 20:15:21 +08:00
pom.xml [SPARK-25592] Setting version to 3.0.0-SNAPSHOT 2018-10-02 08:48:24 -07:00