016e1b491c
## What changes were proposed in this pull request? This PR skip more test when testing with `JAVA_9` or later: 1. Skip `HiveExternalCatalogVersionsSuite` when testing with `JAVA_9` or later because our previous version does not support `JAVA_9` or later. 2. Skip 3 tests in `HiveSparkSubmitSuite` because the `spark.sql.hive.metastore.version` of these tests is lower than `2.0`, however Datanucleus 3.x seem does not support `JAVA_9` or later. Hive upgrade Datanucleus to 4.x from Hive 2.0([HIVE-6113](https://issues.apache.org/jira/browse/HIVE-6113)): ``` [info] Cause: org.datanucleus.exceptions.NucleusException: The java type java.lang.Long (jdbc-type="", sql-type="") cant be mapped for this datastore. No mapping is available. [info] at org.datanucleus.store.rdbms.mapping.RDBMSMappingManager.getDatastoreMappingClass(RDBMSMappingManager.java:1215) [info] at org.datanucleus.store.rdbms.mapping.RDBMSMappingManager.createDatastoreMapping(RDBMSMappingManager.java:1378) [info] at org.datanucleus.store.rdbms.table.AbstractClassTable.addDatastoreId(AbstractClassTable.java:392) [info] at org.datanucleus.store.rdbms.table.ClassTable.initializePK(ClassTable.java:1087) [info] at org.datanucleus.store.rdbms.table.ClassTable.preInitialize(ClassTable.java:247) ``` Please note that this exclude only the tests related to the old metastore library, some other tests of `HiveSparkSubmitSuite` still fail on JDK9+. ## How was this patch tested? manual tests: Test with JDK 11: ``` [info] HiveExternalCatalogVersionsSuite: [info] - backward compatibility !!! CANCELED !!! (37 milliseconds) [info] HiveSparkSubmitSuite: ... [info] - SPARK-8020: set sql conf in spark conf !!! CANCELED !!! (30 milliseconds) [info] org.apache.commons.lang3.SystemUtils.isJavaVersionAtLeast(JAVA_9) was true (HiveSparkSubmitSuite.scala:130) ... [info] - SPARK-9757 Persist Parquet relation with decimal column !!! CANCELED !!! (1 millisecond) [info] org.apache.commons.lang3.SystemUtils.isJavaVersionAtLeast(JAVA_9) was true (HiveSparkSubmitSuite.scala:168) ... [info] - SPARK-16901: set javax.jdo.option.ConnectionURL !!! CANCELED !!! (1 millisecond) [info] org.apache.commons.lang3.SystemUtils.isJavaVersionAtLeast(JAVA_9) was true (HiveSparkSubmitSuite.scala:260) ... ``` Closes #25426 from wangyum/SPARK-28703. Authored-by: Yuming Wang <yumwang@ebay.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> |
||
|---|---|---|
| .. | ||
| benchmarks | ||
| compatibility/src/test/scala/org/apache/spark/sql/hive/execution | ||
| src | ||
| pom.xml | ||