From f50f2d474c31053caf7aeefec7ed499320d09dcc Mon Sep 17 00:00:00 2001 From: Yuanjian Li Date: Fri, 27 Aug 2021 10:27:06 +0900 Subject: [PATCH] [SPARK-35611][SS][FOLLOW-UP] Improve the user guide document ### What changes were proposed in this pull request? Improve the user guide document. ### Why are the changes needed? Make the user guide clear. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Doc change only. Closes #33854 from xuanyuanking/SPARK-35611-follow. Authored-by: Yuanjian Li Signed-off-by: Hyukjin Kwon (cherry picked from commit dd3f0fa8c2318843f04910bb841329d1830b9838) Signed-off-by: Hyukjin Kwon --- docs/structured-streaming-kafka-integration.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/structured-streaming-kafka-integration.md b/docs/structured-streaming-kafka-integration.md index 0ec359f800..6121f19e80 100644 --- a/docs/structured-streaming-kafka-integration.md +++ b/docs/structured-streaming-kafka-integration.md @@ -547,10 +547,10 @@ The following configurations are optional: "error" or "latest" "error" streaming and batch - Defines the behavior when the starting offset by timestamp is specified (either global or per partition), and Kafka doesn't return the matched offset.

+ The strategy will be used when the specified starting offset by timestamp (either global or per partition) doesn't match with the offset Kafka returned. Here's the strategy name and corresponding descriptions:

- "error": fail the query.

- "latest": set the offset to the latest, so that further new records in the partition are being read.

+ "error": fail the query and end users have to deal with workarounds requiring manual steps.

+ "latest": assigns the latest offset for these partitions, so that Spark can read newer records from these partitions in further micro-batches.