[SPARK-16018][SHUFFLE] Shade netty to load shuffle jar in Nodemanger
## What changes were proposed in this pull request? Shade the netty.io namespace so that we can use it in shuffle independent of the dependencies being pulled by hadoop jars. ## How was this patch tested? Ran a decent job involving shuffle write/read and tested the new spark-x-yarn-shuffle jar. After shading netty.io namespace, the nodemanager loads and shuffle job completes successfully. Author: Dhruve Ashar <dhruveashar@gmail.com> Closes #13739 from dhruve/bug/SPARK-16018.
This commit is contained in:
parent
c8809db5a5
commit
298c4ae815
|
@ -96,6 +96,13 @@
|
|||
<include>com.fasterxml.jackson.**</include>
|
||||
</includes>
|
||||
</relocation>
|
||||
<relocation>
|
||||
<pattern>io.netty</pattern>
|
||||
<shadedPattern>${spark.shade.packageName}.io.netty</shadedPattern>
|
||||
<includes>
|
||||
<include>io.netty.**</include>
|
||||
</includes>
|
||||
</relocation>
|
||||
</relocations>
|
||||
</configuration>
|
||||
<executions>
|
||||
|
|
Loading…
Reference in a new issue