History

Daoyuan Wang 92540d22e4 [SPARK-8203] [SPARK-8204] [SQL] conditional function: least/greatest chenghao-intel zhichao-li qiansl127 Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes #6851 from adrian-wang/udflg and squashes the following commits: 0f1bff2 [Daoyuan Wang] address comments from davis 7a6bdbb [Daoyuan Wang] add '.' for hex() c1f6824 [Daoyuan Wang] add codegen, test for all types ec625b0 [Daoyuan Wang] conditional function: least/greatest		2015-07-13 00:14:32 -07:00
..
avro	[SPARK-6123] [SPARK-6775] [SPARK-6776] [SQL] Refactors Parquet read path for interoperability and backwards-compatibility	2015-07-08 15:51:01 -07:00
gen-java/org/apache/spark/sql/parquet/test/avro	[SPARK-8959] [SQL] [HOTFIX] Removes parquet-thrift and libthrift dependencies	2015-07-09 17:09:16 -07:00
java/test/org/apache/spark/sql	[SPARK-7654][SQL] Move JDBC into DataFrame's reader/writer interface.	2015-05-16 22:01:53 -07:00
resources	[SPARK-8959] [SQL] [HOTFIX] Removes parquet-thrift and libthrift dependencies	2015-07-09 17:09:16 -07:00
scala/org/apache/spark/sql	[SPARK-8203] [SPARK-8204] [SQL] conditional function: least/greatest	2015-07-13 00:14:32 -07:00
scripts	[SPARK-6123] [SPARK-6775] [SPARK-6776] [SQL] Refactors Parquet read path for interoperability and backwards-compatibility	2015-07-08 15:51:01 -07:00
thrift	[SPARK-6123] [SPARK-6775] [SPARK-6776] [SQL] Refactors Parquet read path for interoperability and backwards-compatibility	2015-07-08 15:51:01 -07:00
README.md	[SPARK-6123] [SPARK-6775] [SPARK-6776] [SQL] Refactors Parquet read path for interoperability and backwards-compatibility	2015-07-08 15:51:01 -07:00

README.md

Notes for Parquet compatibility tests

The following directories and files are used for Parquet compatibility tests:

.
├── README.md                   # This file
├── avro
│   ├── parquet-compat.avdl     # Testing Avro IDL
│   └── parquet-compat.avpr     # !! NO TOUCH !! Protocol file generated from parquet-compat.avdl
├── gen-java                    # !! NO TOUCH !! Generated Java code
├── scripts
│   └── gen-code.sh             # Script used to generate Java code for Thrift and Avro
└── thrift
    └── parquet-compat.thrift   # Testing Thrift schema

Generated Java code are used in the following test suites:

org.apache.spark.sql.parquet.ParquetAvroCompatibilitySuite
org.apache.spark.sql.parquet.ParquetThriftCompatibilitySuite

To avoid code generation during build time, Java code generated from testing Thrift schema and Avro IDL are also checked in.

When updating the testing Thrift schema and Avro IDL, please run gen-code.sh to update all the generated Java code.

Prerequisites

Please ensure avro-tools and thrift are installed. You may install these two on Mac OS X via:

$ brew install thrift avro-tools