diff --git a/docs/structured-streaming-kafka-integration.md b/docs/structured-streaming-kafka-integration.md index fe3c60040d..b0009e0170 100644 --- a/docs/structured-streaming-kafka-integration.md +++ b/docs/structured-streaming-kafka-integration.md @@ -388,6 +388,16 @@ The following configurations are optional: streaming and batch Rate limit on maximum number of offsets processed per trigger interval. The specified total number of offsets will be proportionally split across topicPartitions of different volume. + + minPartitions + int + none + streaming and batch + Minimum number of partitions to read from Kafka. + By default, Spark has a 1-1 mapping of topicPartitions to Spark partitions consuming from Kafka. + If you set this option to a value greater than your topicPartitions, Spark will divvy up large + Kafka partitions to smaller pieces. + groupIdPrefix string