Imran Rashid acf7ef3154 [SPARK-12297][SQL] Adjust timezone for int96 data from impala ... ## What changes were proposed in this pull request? Int96 data written by impala vs data written by hive & spark is stored slightly differently -- they use a different offset for the timezone. This adds an option "spark.sql.parquet.int96TimestampConversion" (false by default) to adjust timestamps if and only if the writer is impala (or more precisely, if the parquet file's "createdBy" metadata does not start with "parquet-mr"). This matches the existing behavior in hive from HIVE-9482. ## How was this patch tested? Unit test added, existing tests run via jenkins. Author: Imran Rashid <irashid@cloudera.com> Author: Henry Robinson <henry@apache.org> Closes #19769 from squito/SPARK-12297_skip_conversion.		2017-12-09 11:53:15 +09:00
..
main	[SPARK-12297][SQL] Adjust timezone for int96 data from impala	2017-12-09 11:53:15 +09:00
test	[SPARK-22696][SQL] objects functions should not use unneeded global variables	2017-12-07 21:24:36 +08:00