ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/external
History
Max Gekk 26b13c70c3 [SPARK-33169][SQL][TESTS] Check propagation of datasource options to underlying file system for built-in file-based datasources 
### What changes were proposed in this pull request?
1. Add the common trait `CommonFileDataSourceSuite` with tests that can be executed for all built-in file-based datasources.
2. Add a test `CommonFileDataSourceSuite` to check that datasource options are propagated to underlying file systems as Hadoop configs.
3. Mix `CommonFileDataSourceSuite` to `AvroSuite`, `OrcSourceSuite`, `TextSuite`, `JsonSuite`, CSVSuite` and to `ParquetFileFormatSuite`.
4. Remove duplicated tests from `AvroSuite` and from `OrcSourceSuite`.

### Why are the changes needed?
To improve test coverage and test all built-in file-based datasources.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
By running the affected test suites.

Closes #30067 from MaxGekk/ds-options-common-test.

Authored-by: Max Gekk <max.gekk@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
2020-10-19 17:47:49 +09:00
..
avro [SPARK-33169][SQL][TESTS] Check propagation of datasource options to underlying file system for built-in file-based datasources 2020-10-19 17:47:49 +09:00
docker [SPARK-32353][TEST] Update docker/spark-test and clean up unused stuff 2020-07-17 12:05:45 -07:00
docker-integration-tests [SPARK-33129][BUILD][DOCS] Updating the build/sbt references to test-only with testOnly for SBT 1.3.x 2020-10-13 09:21:06 -07:00
kafka-0-10 [SPARK-32873][BUILD] Fix code which causes error when build with sbt and Scala 2.13 2020-09-14 15:34:58 +09:00
kafka-0-10-assembly [SPARK-30950][BUILD] Setting version to 3.1.0-SNAPSHOT 2020-02-25 19:44:31 -08:00
kafka-0-10-sql [SPARK-33139][SQL] protect setActionSession and clearActiveSession 2020-10-16 06:05:17 +00:00
kafka-0-10-token-provider [SPARK-30874][SQL] Support Postgres Kerberos login in JDBC connector 2020-03-12 19:04:35 -07:00
kinesis-asl [SPARK-33079][TESTS] Replace the existing Maven job for Scala 2.13 in Github Actions with SBT job 2020-10-15 20:51:20 +09:00
kinesis-asl-assembly [SPARK-30950][BUILD] Setting version to 3.1.0-SNAPSHOT 2020-02-25 19:44:31 -08:00
spark-ganglia-lgpl [SPARK-30950][BUILD] Setting version to 3.1.0-SNAPSHOT 2020-02-25 19:44:31 -08:00