a133057ce5
## What changes were proposed in this pull request? When reading float4 and smallint columns from PostgreSQL, Spark's `PostgresDialect` widens these types to Decimal and Integer rather than using the narrower Float and Short types. According to https://www.postgresql.org/docs/7.1/static/datatype.html#DATATYPE-TABLE, Postgres maps the `smallint` type to a signed two-byte integer and the `real` / `float4` types to single precision floating point numbers. This patch fixes this by adding more special-cases to `getCatalystType`, similar to what was done for the Derby JDBC dialect. I also fixed a similar problem in the write path which causes Spark to create integer columns in Postgres for what should have been ShortType columns. ## How was this patch tested? New test cases in `PostgresIntegrationSuite` (which I ran manually because Jenkins can't run it right now). Author: Josh Rosen <joshrosen@databricks.com> Closes #14796 from JoshRosen/postgres-jdbc-type-fixes. |
||
---|---|---|
.. | ||
docker | ||
docker-integration-tests | ||
flume | ||
flume-assembly | ||
flume-sink | ||
java8-tests | ||
kafka-0-8 | ||
kafka-0-8-assembly | ||
kafka-0-10 | ||
kafka-0-10-assembly | ||
kinesis-asl | ||
kinesis-asl-assembly | ||
spark-ganglia-lgpl |