spark-instrumented-optimizer/external
Shixiong Zhu b9c2521de2 [SPARK-28489][SS] Fix a bug that KafkaOffsetRangeCalculator.getRanges may drop offsets
## What changes were proposed in this pull request?

`KafkaOffsetRangeCalculator.getRanges` may drop offsets due to round off errors. The test added in this PR is one example.

This PR rewrites the logic in `KafkaOffsetRangeCalculator.getRanges` to ensure it never drops offsets.

## How was this patch tested?

The regression test.

Closes #25237 from zsxwing/fix-range.

Authored-by: Shixiong Zhu <zsxwing@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2019-07-26 00:10:56 -07:00
..
avro [SPARK-28469][SQL] Change CalendarIntervalType's readable string representation from calendarinterval to interval 2019-07-22 20:53:59 -07:00
docker [SPARK-27794][R][DOCS] Use https URL for CRAN repo 2019-05-22 14:28:21 -07:00
docker-integration-tests [SPARK-28097][SQL] Map ByteType to SMALLINT for PostgresDialect 2019-07-17 15:10:01 -07:00
kafka-0-10 [SPARK-28335][DSTREAMS][TEST] DirectKafkaStreamSuite wait for Kafka async commit 2019-07-10 09:35:39 -07:00
kafka-0-10-assembly [SPARK-25956] Make Scala 2.12 as default Scala version in Spark 3.0 2018-11-14 16:22:23 -08:00
kafka-0-10-sql [SPARK-28489][SS] Fix a bug that KafkaOffsetRangeCalculator.getRanges may drop offsets 2019-07-26 00:10:56 -07:00
kafka-0-10-token-provider [SPARK-28055][SS][DSTREAMS] Add delegation token custom AdminClient configurations. 2019-07-11 09:36:24 -07:00
kinesis-asl [SPARK-27610][YARN] Shade netty native libraries 2019-05-07 10:47:36 -07:00
kinesis-asl-assembly [SPARK-25956] Make Scala 2.12 as default Scala version in Spark 3.0 2018-11-14 16:22:23 -08:00
spark-ganglia-lgpl [SPARK-25956] Make Scala 2.12 as default Scala version in Spark 3.0 2018-11-14 16:22:23 -08:00