ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/sql/core
History
Yuming Wang 33f3c48cac [SPARK-27176][SQL] Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4 
## What changes were proposed in this pull request?

This PR mainly contains:
1. Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4.
2. Resolve compatibility issues between Hive 1.2.1 and Hive 2.3.4 in the `sql/hive` module.

## How was this patch tested?
jenkins test hadoop-2.7
manual test hadoop-3:
```shell
build/sbt clean package -Phadoop-3.2 -Phive
export SPARK_PREPEND_CLASSES=true

# rm -rf metastore_db

cat <<EOF > test_hadoop3.scala
spark.range(10).write.saveAsTable("test_hadoop3")
spark.table("test_hadoop3").show
EOF

bin/spark-shell --conf spark.hadoop.hive.metastore.schema.verification=false --conf spark.hadoop.datanucleus.schema.autoCreateAll=true -i test_hadoop3.scala
```

Closes #23788 from wangyum/SPARK-23710-hadoop3.

Authored-by: Yuming Wang <yumwang@ebay.com>
Signed-off-by: gatorsmile <gatorsmile@gmail.com>
2019-04-08 08:42:21 -07:00
..
benchmarks [SPARK-27327][SQL] New JSON benchmarks: functions, Dataset[String] 2019-04-01 08:33:16 +09:00
src [SPARK-25407][SQL] Allow nested access for non-existent field for Parquet file when nested pruning is enabled 2019-04-08 22:26:02 +09:00
v1.2.1/src [SPARK-27182][SQL] Move the conflict source code of the sql/core module to sql/core/v1.2.1 2019-03-26 22:32:03 -07:00
v2.3.4/src [SPARK-27176][SQL] Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4 2019-04-08 08:42:21 -07:00
pom.xml [SPARK-27182][SQL] Move the conflict source code of the sql/core module to sql/core/v1.2.1 2019-03-26 22:32:03 -07:00