e9c36938ba
In order for this to work, I had to disable gap sampling. Author: Reynold Xin <rxin@databricks.com> Closes #8040 from rxin/SPARK-9752 and squashes the following commits: f9e248c [Reynold Xin] Fix the test case for real this time. adbccb3 [Reynold Xin] Fixed test case. 589fb23 [Reynold Xin] Merge branch 'SPARK-9752' of github.com:rxin/spark into SPARK-9752 55ccddc [Reynold Xin] Fixed core test. 78fa895 [Reynold Xin] [SPARK-9752][SQL] Support UnsafeRow in Sample operator. c9e7112 [Reynold Xin] [SPARK-9752][SQL] Support UnsafeRow in Sample operator. |
||
|---|---|---|
| .. | ||
| avro | ||
| gen-java/org/apache/spark/sql/parquet/test/avro | ||
| java/test/org/apache/spark/sql | ||
| resources | ||
| scala/org/apache/spark/sql | ||
| scripts | ||
| thrift | ||
| README.md | ||
Notes for Parquet compatibility tests
The following directories and files are used for Parquet compatibility tests:
.
├── README.md # This file
├── avro
│ ├── parquet-compat.avdl # Testing Avro IDL
│ └── parquet-compat.avpr # !! NO TOUCH !! Protocol file generated from parquet-compat.avdl
├── gen-java # !! NO TOUCH !! Generated Java code
├── scripts
│ └── gen-code.sh # Script used to generate Java code for Thrift and Avro
└── thrift
└── parquet-compat.thrift # Testing Thrift schema
Generated Java code are used in the following test suites:
org.apache.spark.sql.parquet.ParquetAvroCompatibilitySuiteorg.apache.spark.sql.parquet.ParquetThriftCompatibilitySuite
To avoid code generation during build time, Java code generated from testing Thrift schema and Avro IDL are also checked in.
When updating the testing Thrift schema and Avro IDL, please run gen-code.sh to update all the generated Java code.
Prerequisites
Please ensure avro-tools and thrift are installed. You may install these two on Mac OS X via:
$ brew install thrift avro-tools