spark-instrumented-optimizer/sql/core/benchmarks
Wenchen Fan 34f229bc21 [SPARK-25710][SQL] range should report metrics correctly
## What changes were proposed in this pull request?

Currently `Range` reports metrics in batch granularity. This is acceptable, but it's better if we can make it row granularity without performance penalty.

Before this PR,  the metrics are updated when preparing the batch, which is before we actually consume data. In this PR, the metrics are updated after the data are consumed. There are 2 different cases:
1. The data processing loop has a stop check. The metrics are updated when we need to stop.
2. no stop check. The metrics are updated after the loop.

## How was this patch tested?

existing tests and a new benchmark

Closes #22698 from cloud-fan/range.

Authored-by: Wenchen Fan <wenchen@databricks.com>
Signed-off-by: Wenchen Fan <wenchen@databricks.com>
2018-10-13 13:55:28 +08:00
..
AggregateBenchmark-results.txt [SPARK-25476][SPARK-25510][TEST] Refactor AggregateBenchmark and add a new trait to better support Dataset and DataFrame API 2018-10-01 07:32:40 -07:00
BloomFilterBenchmark-results.txt [SPARK-25589][SQL][TEST] Add BloomFilterBenchmark 2018-10-03 04:14:07 -07:00
ColumnarBatchBenchmark-results.txt [SPARK-25481][SQL][TEST] Refactor ColumnarBatchBenchmark to use main method 2018-09-26 20:40:10 -07:00
CompressionSchemeBenchmark-results.txt [SPARK-25478][SQL][TEST] Refactor CompressionSchemeBenchmark to use main method 2018-09-23 20:46:40 -07:00
DatasetBenchmark-results.txt [SPARK-25479][TEST] Refactor DatasetBenchmark to use main method 2018-10-04 11:58:16 -07:00
DataSourceReadBenchmark-results.txt [SPARK-25662][SQL][TEST] Refactor DataSourceReadBenchmark to use main method 2018-10-11 20:27:07 +00:00
FilterPushdownBenchmark-results.txt [SPARK-25438][SQL][TEST] Fix FilterPushdownBenchmark to use the same memory assumption 2018-09-15 17:48:39 -07:00
JoinBenchmark-results.txt [SPARK-25664][SQL][TEST] Refactor JoinBenchmark to use main method 2018-10-12 16:08:12 -07:00
MiscBenchmark-results.txt [SPARK-25488][SQL][TEST] Refactor MiscBenchmark to use main method 2018-10-06 08:47:43 -07:00
PrimitiveArrayBenchmark-results.txt [SPARK-25487][SQL][TEST] Refactor PrimitiveArrayBenchmark 2018-09-21 15:04:47 +09:00
RangeBenchmark-results.txt [SPARK-25710][SQL] range should report metrics correctly 2018-10-13 13:55:28 +08:00
SortBenchmark-results.txt [SPARK-25486][TEST] Refactor SortBenchmark to use main method 2018-09-25 11:13:05 -07:00
UnsafeArrayDataBenchmark-results.txt [SPARK-25483][TEST] Refactor UnsafeArrayDataBenchmark to use main method 2018-10-03 04:20:02 -07:00
WideSchemaBenchmark-results.txt [SPARK-17335][SQL] Fix ArrayType and MapType CatalogString. 2016-09-03 19:02:20 +02:00