[SPARK-17875][CORE][BUILD] Remove dependency on Netty 3
### What changes were proposed in this pull request? Spark uses Netty 4 directly, but also includes Netty 3 only because transitive dependencies do. The dependencies (Hadoop HDFS, Zookeeper, Avro) don't seem to need this dependency as used in Spark. I think we can forcibly remove it to slim down the dependencies. Previous attempts were blocked by its usage in Flume, but that dependency has gone away. https://github.com/apache/spark/pull/15436 ### Why are the changes needed? Mostly to reduce the transitive dependency size and complexity a little bit and avoid triggering spurious security alerts on Netty 3.x usage. ### Does this PR introduce any user-facing change? No ### How was this patch tested? Existing tests Closes #25544 from srowen/SPARK-17875. Authored-by: Sean Owen <sean.owen@databricks.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
This commit is contained in:
Sean Owen
Dongjoon Hyun
parent
aefb2e70e7
commit
9ea37b09cf
|
|
@ -253,7 +253,6 @@ commons-codec:commons-codec
|
|||
commons-collections:commons-collections
|
||||
io.fabric8:kubernetes-client
|
||||
io.fabric8:kubernetes-model
|
||||
io.netty:netty
|
||||
io.netty:netty-all
|
||||
net.hydromatic:eigenbase-properties
|
||||
net.sf.supercsv:super-csv
|
||||
|
|
|
|||
|
|
@ -95,6 +95,7 @@ The binary distribution of this product bundles binaries of
|
|||
Gson 2.2.4,
|
||||
which has the following notices:
|
||||
|
||||
|
||||
The Netty Project
|
||||
=================
|
||||
|
||||
|
|
@ -154,29 +155,16 @@ facade for Java, which can be obtained at:
|
|||
* HOMEPAGE:
|
||||
* http://www.slf4j.org/
|
||||
|
||||
This product contains a modified portion of 'ArrayDeque', written by Josh
|
||||
Bloch of Google, Inc:
|
||||
|
||||
* LICENSE:
|
||||
* license/LICENSE.deque.txt (Public Domain)
|
||||
|
||||
This product contains a modified portion of 'Apache Harmony', an open source
|
||||
Java SE, which can be obtained at:
|
||||
|
||||
* NOTICE:
|
||||
* license/NOTICE.harmony.txt
|
||||
* LICENSE:
|
||||
* license/LICENSE.harmony.txt (Apache License 2.0)
|
||||
* HOMEPAGE:
|
||||
* http://archive.apache.org/dist/harmony/
|
||||
|
||||
This product contains a modified version of Roland Kuhn's ASL2
|
||||
AbstractNodeQueue, which is based on Dmitriy Vyukov's non-intrusive MPSC queue.
|
||||
It can be obtained at:
|
||||
|
||||
* LICENSE:
|
||||
* license/LICENSE.abstractnodequeue.txt (Public Domain)
|
||||
* HOMEPAGE:
|
||||
* https://github.com/akka/akka/blob/wip-2.2.3-for-scala-2.11/akka-actor/src/main/java/akka/dispatch/AbstractNodeQueue.java
|
||||
|
||||
This product contains a modified portion of 'jbzip2', a Java bzip2 compression
|
||||
and decompression library written by Matthew J. Francis. It can be obtained at:
|
||||
|
||||
|
|
@ -192,7 +180,7 @@ a constant-size alphabet written by Yuta Mori. It can be obtained at:
|
|||
* LICENSE:
|
||||
* license/LICENSE.libdivsufsort.txt (MIT License)
|
||||
* HOMEPAGE:
|
||||
* https://code.google.com/p/libdivsufsort/
|
||||
* https://github.com/y-256/libdivsufsort
|
||||
|
||||
This product contains a modified portion of Nitsan Wakart's 'JCTools', Java Concurrency Tools for the JVM,
|
||||
which can be obtained at:
|
||||
|
|
@ -248,7 +236,7 @@ interchange format, which can be obtained at:
|
|||
* LICENSE:
|
||||
* license/LICENSE.protobuf.txt (New BSD License)
|
||||
* HOMEPAGE:
|
||||
* http://code.google.com/p/protobuf/
|
||||
* https://github.com/google/protobuf
|
||||
|
||||
This product optionally depends on 'Bouncy Castle Crypto APIs' to generate
|
||||
a temporary self-signed X.509 certificate when the JVM does not provide the
|
||||
|
|
@ -265,7 +253,7 @@ by Google Inc, which can be obtained at:
|
|||
* LICENSE:
|
||||
* license/LICENSE.snappy.txt (New BSD License)
|
||||
* HOMEPAGE:
|
||||
* http://code.google.com/p/snappy/
|
||||
* https://github.com/google/snappy
|
||||
|
||||
This product optionally depends on 'JBoss Marshalling', an alternative Java
|
||||
serialization API, which can be obtained at:
|
||||
|
|
@ -281,7 +269,7 @@ benchmarking framework, which can be obtained at:
|
|||
* LICENSE:
|
||||
* license/LICENSE.caliper.txt (Apache License 2.0)
|
||||
* HOMEPAGE:
|
||||
* http://code.google.com/p/caliper/
|
||||
* https://github.com/google/caliper
|
||||
|
||||
This product optionally depends on 'Apache Commons Logging', a logging
|
||||
framework, which can be obtained at:
|
||||
|
|
@ -323,6 +311,15 @@ provides utilities for the java.lang API, which can be obtained at:
|
|||
* HOMEPAGE:
|
||||
* https://commons.apache.org/proper/commons-lang/
|
||||
|
||||
|
||||
This product contains the Maven wrapper scripts from 'Maven Wrapper', that provides an easy way to ensure a user has everything necessary to run the Maven build.
|
||||
|
||||
* LICENSE:
|
||||
* license/LICENSE.mvn-wrapper.txt (Apache License 2.0)
|
||||
* HOMEPAGE:
|
||||
* https://github.com/takari/maven-wrapper
|
||||
|
||||
|
||||
The binary distribution of this product bundles binaries of
|
||||
Commons Codec 1.4,
|
||||
which has the following notices:
|
||||
|
|
@ -619,45 +616,6 @@ fixes; these can be found from file "VERSION.txt" in SCM.
|
|||
Apache Commons Net
|
||||
Copyright 2001-2012 The Apache Software Foundation
|
||||
|
||||
Copyright 2011 The Netty Project
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
This product contains a modified version of 'JZlib', a re-implementation of
|
||||
zlib in pure Java, which can be obtained at:
|
||||
|
||||
* LICENSE:
|
||||
* license/LICENSE.jzlib.txt (BSD Style License)
|
||||
* HOMEPAGE:
|
||||
* http://www.jcraft.com/jzlib/
|
||||
|
||||
This product contains a modified version of 'Webbit', a Java event based
|
||||
WebSocket and HTTP server:
|
||||
|
||||
This product optionally depends on 'Protocol Buffers', Google's data
|
||||
interchange format, which can be obtained at:
|
||||
|
||||
This product optionally depends on 'SLF4J', a simple logging facade for Java,
|
||||
which can be obtained at:
|
||||
|
||||
This product optionally depends on 'Apache Log4J', a logging framework,
|
||||
which can be obtained at:
|
||||
|
||||
This product optionally depends on 'JBoss Logging', a logging framework,
|
||||
which can be obtained at:
|
||||
|
||||
* LICENSE:
|
||||
* license/LICENSE.jboss-logging.txt (GNU LGPL 2.1)
|
||||
* HOMEPAGE:
|
||||
* http://anonsvn.jboss.org/repos/common/common-logging-spi/
|
||||
|
||||
This product optionally depends on 'Apache Felix', an open source OSGi
|
||||
framework implementation, which can be obtained at:
|
||||
|
||||
* LICENSE:
|
||||
* license/LICENSE.felix.txt (Apache License 2.0)
|
||||
* HOMEPAGE:
|
||||
* http://felix.apache.org/
|
||||
|
||||
Jackson core and extension components may be licensed under different licenses.
|
||||
To find the details that apply to this artifact see the accompanying LICENSE file.
|
||||
|
|
|
|||
|
|
@ -273,10 +273,6 @@
|
|||
<groupId>io.netty</groupId>
|
||||
<artifactId>netty-all</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.netty</groupId>
|
||||
<artifactId>netty</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.clearspring.analytics</groupId>
|
||||
<artifactId>stream</artifactId>
|
||||
|
|
|
|||
|
|
@ -149,7 +149,6 @@ metrics-graphite-3.1.5.jar
|
|||
metrics-json-3.1.5.jar
|
||||
metrics-jvm-3.1.5.jar
|
||||
minlog-1.3.0.jar
|
||||
netty-3.9.9.Final.jar
|
||||
netty-all-4.1.30.Final.jar
|
||||
objenesis-2.5.1.jar
|
||||
okapi-shade-0.4.2.jar
|
||||
|
|
|
|||
|
|
@ -166,7 +166,6 @@ metrics-json-3.1.5.jar
|
|||
metrics-jvm-3.1.5.jar
|
||||
minlog-1.3.0.jar
|
||||
mssql-jdbc-6.2.1.jre7.jar
|
||||
netty-3.9.9.Final.jar
|
||||
netty-all-4.1.30.Final.jar
|
||||
nimbus-jose-jwt-4.41.1.jar
|
||||
objenesis-2.5.1.jar
|
||||
|
|
|
|||
13
pom.xml
13
pom.xml
|
|
@ -658,11 +658,6 @@
|
|||
<artifactId>netty-all</artifactId>
|
||||
<version>4.1.30.Final</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>io.netty</groupId>
|
||||
<artifactId>netty</artifactId>
|
||||
<version>3.9.9.Final</version>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.apache.derby</groupId>
|
||||
<artifactId>derby</artifactId>
|
||||
|
|
@ -979,6 +974,10 @@
|
|||
<groupId>org.jboss.netty</groupId>
|
||||
<artifactId>netty</artifactId>
|
||||
</exclusion>
|
||||
<exclusion>
|
||||
<groupId>io.netty</groupId>
|
||||
<artifactId>netty</artifactId>
|
||||
</exclusion>
|
||||
<exclusion>
|
||||
<!-- BeanUtils >= 1.9.0 no longer splits out -core; exclude it -->
|
||||
<groupId>commons-beanutils</groupId>
|
||||
|
|
@ -1326,6 +1325,10 @@
|
|||
<groupId>jline</groupId>
|
||||
<artifactId>jline</artifactId>
|
||||
</exclusion>
|
||||
<exclusion>
|
||||
<groupId>io.netty</groupId>
|
||||
<artifactId>netty</artifactId>
|
||||
</exclusion>
|
||||
</exclusions>
|
||||
</dependency>
|
||||
<dependency>
|
||||
|
|
|
|||
Loading…
Reference in a new issue