History

Tathagata Das bde27b89a2 [SPARK-15131][SQL] Shutdown StateStore management thread when SparkContext has been shutdown ## What changes were proposed in this pull request? Make sure that whenever the StateStoreCoordinator cannot be contacted, assume that the SparkContext and RpcEnv on the driver has been shutdown, and therefore stop the StateStore management thread, and unload all loaded stores. ## How was this patch tested? Updated unit tests. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #12905 from tdas/SPARK-15131.		2016-05-04 21:19:53 -07:00
..
avro	[SPARK-10136] [SQL] Fixes Parquet support for Avro array of primitive array	2015-08-20 11:00:29 -07:00
gen-java/org/apache/spark/sql/execution/datasources/parquet/test/avro	[SPARK-13401][SQL][TESTS] Fix SQL test warnings.	2016-03-22 21:08:11 -07:00
java/test/org/apache/spark/sql	[SPARK-12660][SPARK-14967][SQL] Implement Except Distinct by Left Anti Join	2016-04-29 15:30:36 +08:00
resources	[SPARK-14143] Options for parsing NaNs, Infinity and nulls for numeric types	2016-04-30 18:12:03 -07:00
scala/org/apache/spark/sql	[SPARK-15131][SQL] Shutdown StateStore management thread when SparkContext has been shutdown	2016-05-04 21:19:53 -07:00
scripts	[SPARK-9407] [SQL] Relaxes Parquet ValidTypeMap to allow ENUM predicates to be pushed down	2015-08-12 20:01:34 +08:00
thrift	[SPARK-9407] [SQL] Relaxes Parquet ValidTypeMap to allow ENUM predicates to be pushed down	2015-08-12 20:01:34 +08:00
README.md	[SPARK-9407] [SQL] Relaxes Parquet ValidTypeMap to allow ENUM predicates to be pushed down	2015-08-12 20:01:34 +08:00

README.md

Notes for Parquet compatibility tests

The following directories and files are used for Parquet compatibility tests:

.
├── README.md                   # This file
├── avro
│   ├── *.avdl                  # Testing Avro IDL(s)
│   └── *.avpr                  # !! NO TOUCH !! Protocol files generated from Avro IDL(s)
├── gen-java                    # !! NO TOUCH !! Generated Java code
├── scripts
│   ├── gen-avro.sh             # Script used to generate Java code for Avro
│   └── gen-thrift.sh           # Script used to generate Java code for Thrift
└── thrift
    └── *.thrift                # Testing Thrift schema(s)

To avoid code generation during build time, Java code generated from testing Thrift schema and Avro IDL are also checked in.

When updating the testing Thrift schema and Avro IDL, please run gen-avro.sh and gen-thrift.sh accordingly to update generated Java code.

Prerequisites

Please ensure avro-tools and thrift are installed. You may install these two on Mac OS X via:

$ brew install thrift avro-tools