0a73aa31f4
## What changes were proposed in this pull request? Migrating KafkaSource (with data source v1) to KafkaMicroBatchReader (with data source v2). Performance comparison: In a unit test with in-process Kafka broker, I tested the read throughput of V1 and V2 using 20M records in a single partition. They were comparable. ## How was this patch tested? Existing tests, few modified to be better tests than the existing ones. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #20554 from tdas/SPARK-23362.
109 lines
1.6 KiB
Plaintext
109 lines
1.6 KiB
Plaintext
target
|
|
cache
|
|
.gitignore
|
|
.gitattributes
|
|
.project
|
|
.classpath
|
|
.mima-excludes
|
|
.generated-mima-excludes
|
|
.generated-mima-class-excludes
|
|
.generated-mima-member-excludes
|
|
.rat-excludes
|
|
.*md
|
|
derby.log
|
|
TAGS
|
|
RELEASE
|
|
control
|
|
docs
|
|
slaves
|
|
spark-env.cmd
|
|
bootstrap-tooltip.js
|
|
jquery-1.11.1.min.js
|
|
d3.min.js
|
|
dagre-d3.min.js
|
|
graphlib-dot.min.js
|
|
sorttable.js
|
|
vis.min.js
|
|
vis.min.css
|
|
dataTables.bootstrap.css
|
|
dataTables.bootstrap.min.js
|
|
dataTables.rowsGroup.js
|
|
jquery.blockUI.min.js
|
|
jquery.cookies.2.2.0.min.js
|
|
jquery.dataTables.1.10.4.min.css
|
|
jquery.dataTables.1.10.4.min.js
|
|
jquery.mustache.js
|
|
jsonFormatter.min.css
|
|
jsonFormatter.min.js
|
|
.*avsc
|
|
.*txt
|
|
.*json
|
|
.*data
|
|
.*log
|
|
cloudpickle.py
|
|
heapq3.py
|
|
join.py
|
|
SparkExprTyper.scala
|
|
SparkILoop.scala
|
|
SparkILoopInit.scala
|
|
SparkIMain.scala
|
|
SparkImports.scala
|
|
SparkJLineCompletion.scala
|
|
SparkJLineReader.scala
|
|
SparkMemberHandlers.scala
|
|
SparkReplReporter.scala
|
|
sbt
|
|
sbt-launch-lib.bash
|
|
plugins.sbt
|
|
work
|
|
.*\.q
|
|
.*\.qv
|
|
golden
|
|
test.out/*
|
|
.*iml
|
|
service.properties
|
|
db.lck
|
|
build/*
|
|
dist/*
|
|
.*out
|
|
.*ipr
|
|
.*iws
|
|
logs
|
|
.*scalastyle-output.xml
|
|
.*dependency-reduced-pom.xml
|
|
known_translations
|
|
json_expectation
|
|
app-20180109111548-0000
|
|
app-20161115172038-0000
|
|
app-20161116163331-0000
|
|
application_1516285256255_0012
|
|
local-1422981759269
|
|
local-1422981780767
|
|
local-1425081759269
|
|
local-1426533911241
|
|
local-1426633911242
|
|
local-1430917381534
|
|
local-1430917381535_1
|
|
local-1430917381535_2
|
|
DESCRIPTION
|
|
NAMESPACE
|
|
test_support/*
|
|
.*Rd
|
|
help/*
|
|
html/*
|
|
INDEX
|
|
.lintr
|
|
gen-java.*
|
|
.*avpr
|
|
.*parquet
|
|
spark-deps-.*
|
|
.*csv
|
|
.*tsv
|
|
.*\.sql
|
|
.Rbuildignore
|
|
META-INF/*
|
|
spark-warehouse
|
|
structured-streaming/*
|
|
kafka-source-initial-offset-version-2.1.0.bin
|
|
kafka-source-initial-offset-future-version.bin
|