spark-instrumented-optimizer/external
Yuming Wang 17edfec59d [SPARK-20427][SQL] Read JDBC table use custom schema
## What changes were proposed in this pull request?

Auto generated Oracle schema some times not we expect:

- `number(1)` auto mapped to BooleanType, some times it's not we expect, per [SPARK-20921](https://issues.apache.org/jira/browse/SPARK-20921).
-  `number` auto mapped to Decimal(38,10), It can't read big data, per [SPARK-20427](https://issues.apache.org/jira/browse/SPARK-20427).

This PR fix this issue by custom schema as follows:
```scala
val props = new Properties()
props.put("customSchema", "ID decimal(38, 0), N1 int, N2 boolean")
val dfRead = spark.read.schema(schema).jdbc(jdbcUrl, "tableWithCustomSchema", props)
dfRead.show()
```
or
```sql
CREATE TEMPORARY VIEW tableWithCustomSchema
USING org.apache.spark.sql.jdbc
OPTIONS (url '$jdbcUrl', dbTable 'tableWithCustomSchema', customSchema'ID decimal(38, 0), N1 int, N2 boolean')
```

## How was this patch tested?

unit tests

Author: Yuming Wang <wgyumg@gmail.com>

Closes #18266 from wangyum/SPARK-20427.
2017-09-13 16:34:17 -07:00
..
docker [SPARK-20759] SCALA_VERSION in _config.yml should be consistent with pom.xml 2017-05-19 15:26:39 +01:00
docker-integration-tests [SPARK-20427][SQL] Read JDBC table use custom schema 2017-09-13 16:34:17 -07:00
flume [SPARK-21731][BUILD] Upgrade scalastyle to 0.9. 2017-08-15 13:59:00 -07:00
flume-assembly [SPARK-20453] Bump master branch version to 2.3.0-SNAPSHOT 2017-04-24 21:48:04 -07:00
flume-sink [SPARK-20453] Bump master branch version to 2.3.0-SNAPSHOT 2017-04-24 21:48:04 -07:00
kafka-0-8 [SPARK-21893][BUILD][STREAMING][WIP] Put Kafka 0.8 behind a profile 2017-09-13 10:10:40 +01:00
kafka-0-8-assembly [SPARK-21276][CORE] Update lz4-java to the latest (v1.4.0) 2017-08-09 17:31:52 +02:00
kafka-0-10 [MINOR][SQL] remove unuse import class 2017-09-11 10:09:20 +01:00
kafka-0-10-assembly [SPARK-21276][CORE] Update lz4-java to the latest (v1.4.0) 2017-08-09 17:31:52 +02:00
kafka-0-10-sql [SPARK-21873][SS] - Avoid using return inside CachedKafkaConsumer.get 2017-08-30 10:33:23 +01:00
kinesis-asl [SPARK-14280][BUILD][WIP] Update change-version.sh and pom.xml to add Scala 2.12 profiles and enable 2.12 compilation 2017-09-01 19:21:21 +01:00
kinesis-asl-assembly [SPARK-20453] Bump master branch version to 2.3.0-SNAPSHOT 2017-04-24 21:48:04 -07:00
spark-ganglia-lgpl [SPARK-20453] Bump master branch version to 2.3.0-SNAPSHOT 2017-04-24 21:48:04 -07:00