spark-instrumented-optimizer/external
Gengliang Wang ab197308a7
[SPARK-25104][SQL] Avro: Validate user specified output schema
## What changes were proposed in this pull request?

With code changes in https://github.com/apache/spark/pull/21847 , Spark can write out to Avro file as per user provided output schema.

To make it more robust and user friendly, we should validate the Avro schema before tasks launched.

Also we should support output logical decimal type as BYTES (By default we output as FIXED)

## How was this patch tested?

Unit test

Closes #22094 from gengliangwang/AvroSerializerMatch.

Authored-by: Gengliang Wang <gengliang.wang@databricks.com>
Signed-off-by: DB Tsai <d_tsai@apple.com>
2018-08-14 04:43:14 +00:00
..
avro [SPARK-25104][SQL] Avro: Validate user specified output schema 2018-08-14 04:43:14 +00:00
docker [SPARK-23038][TEST] Update docker/spark-test (JDK/OS) 2018-01-13 23:26:12 -08:00
docker-integration-tests [SPARK-22814][SQL] Support Date/Timestamp in a JDBC partition column 2018-07-30 07:42:00 -07:00
flume [SPARK-23028] Bump master branch version to 2.4.0-SNAPSHOT 2018-01-13 00:37:59 +08:00
flume-assembly [SPARK-23028] Bump master branch version to 2.4.0-SNAPSHOT 2018-01-13 00:37:59 +08:00
flume-sink [SPARK-23028] Bump master branch version to 2.4.0-SNAPSHOT 2018-01-13 00:37:59 +08:00
kafka-0-8 [SPARK-21168] KafkaRDD should always set kafka clientId. 2018-04-23 13:56:11 -05:00
kafka-0-8-assembly [SPARK-23028] Bump master branch version to 2.4.0-SNAPSHOT 2018-01-13 00:37:59 +08:00
kafka-0-10 [SPARK-18057][FOLLOW-UP][SS] Update Kafka client version from 0.10.0.1 to 2.0.0 2018-08-03 16:22:54 -07:00
kafka-0-10-assembly [SPARK-23028] Bump master branch version to 2.4.0-SNAPSHOT 2018-01-13 00:37:59 +08:00
kafka-0-10-sql [MINOR][DOC] Fix typo 2018-08-09 20:10:17 +08:00
kinesis-asl [SPARK-20168][STREAMING KINESIS] Setting the timestamp directly would cause exception on … 2018-07-12 10:04:47 -07:00
kinesis-asl-assembly [SPARK-23028] Bump master branch version to 2.4.0-SNAPSHOT 2018-01-13 00:37:59 +08:00
spark-ganglia-lgpl [SPARK-23028] Bump master branch version to 2.4.0-SNAPSHOT 2018-01-13 00:37:59 +08:00