ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-34112][BUILD] Upgrade ORC to 1.7.0

Browse source 
### What changes were proposed in this pull request?

This PR aims to upgrade Apache ORC from 1.6.11 to 1.7.0 for Apache Spark 3.3.0.

### Why are the changes needed?

[Apache ORC 1.7.0](https://orc.apache.org/news/2021/09/15/ORC-1.7.0/) is a new release with the following new features and improvements.
  - ORC-377 Support Snappy compression in C++ Writer
  - ORC-577 Support row-level filtering
  - ORC-716 Build and test on Java 17-EA
  - ORC-731 Improve Java Tools
  - ORC-742 LazyIO of non-filter columns
  - ORC-751 Implement Predicate Pushdown in C++ Reader
  - ORC-755 Introduce OrcFilterContext
  - ORC-757 Add Hashtable implementation for dictionary
  - ORC-780 Support LZ4 Compression in C++ Writer
  - ORC-797 Allow writers to get the stripe information
  - ORC-818 Build and test in Apple Silicon
  - ORC-861 Bump CMake minimum requirement to 2.8.12
  - ORC-867 Upgrade hive-storage-api to 2.8.1
  - ORC-984 Save the software version that wrote each ORC file

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Pass the existing CIs because this is a dependency change.

Closes #34045 from dongjoon-hyun/SPARK-34112.

Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This commit is contained in:
Dongjoon Hyun 2021-09-20 01:09:15 -07:00
parent 8d8b4aabc3
commit a396dd6216
3 changed files with 7 additions and 7 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
dev/deps/spark-deps-hadoop-2.7-hive-2.3
View file

@ -195,9 +195,9 @@ objenesis/2.6//objenesis-2.6.jar
okhttp/3.12.12//okhttp-3.12.12.jar
okio/1.14.0//okio-1.14.0.jar
opencsv/2.3//opencsv-2.3.jar
orc-core/1.6.11//orc-core-1.6.11.jar
orc-mapreduce/1.6.11//orc-mapreduce-1.6.11.jar
orc-shims/1.6.11//orc-shims-1.6.11.jar
orc-core/1.7.0//orc-core-1.7.0.jar
orc-mapreduce/1.7.0//orc-mapreduce-1.7.0.jar
orc-shims/1.7.0//orc-shims-1.7.0.jar
oro/2.0.8//oro-2.0.8.jar
osgi-resource-locator/1.0.3//osgi-resource-locator-1.0.3.jar
paranamer/2.8//paranamer-2.8.jar

6
dev/deps/spark-deps-hadoop-3.2-hive-2.3
View file

@ -165,9 +165,9 @@ objenesis/2.6//objenesis-2.6.jar
okhttp/3.12.12//okhttp-3.12.12.jar
okio/1.14.0//okio-1.14.0.jar
opencsv/2.3//opencsv-2.3.jar
orc-core/1.6.11//orc-core-1.6.11.jar
orc-mapreduce/1.6.11//orc-mapreduce-1.6.11.jar
orc-shims/1.6.11//orc-shims-1.6.11.jar
orc-core/1.7.0//orc-core-1.7.0.jar
orc-mapreduce/1.7.0//orc-mapreduce-1.7.0.jar
orc-shims/1.7.0//orc-shims-1.7.0.jar
oro/2.0.8//oro-2.0.8.jar
osgi-resource-locator/1.0.3//osgi-resource-locator-1.0.3.jar
paranamer/2.8//paranamer-2.8.jar

2
pom.xml
View file

@ -137,7 +137,7 @@
<!-- After 10.15.1.3, the minimum required version is JDK9 -->
<derby.version>10.14.2.0</derby.version>
<parquet.version>1.12.1</parquet.version>
<orc.version>1.6.11</orc.version>
<orc.version>1.7.0</orc.version>
<jetty.version>9.4.43.v20210629</jetty.version>
<jakartaservlet.version>4.0.3</jakartaservlet.version>
<chill.version>0.10.0</chill.version>