9aa65a6e0e
### What changes were proposed in this pull request? This PR fixes an issue that `YarnClusterSuite` fails due to `NoClassDefFoundError unless `hadoop-3.2` profile is activated explicitly regardless of building with SBT or Maven. ``` build/sbt -Pyarn "yarn/testOnly org.apache.spark.deploy.yarn.YarnClusterSuite" ... [info] YarnClusterSuite: [info] org.apache.spark.deploy.yarn.YarnClusterSuite *** ABORTED *** (598 milliseconds) [info] java.lang.NoClassDefFoundError: org/bouncycastle/operator/OperatorCreationException [info] at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:888) [info] at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) [info] at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1410) [info] at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:344) [info] at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164) [info] at org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:359) ``` The solution is modifying `yarn/pom.xml` to activate `hadoop-3.2` profiles by default. ### Why are the changes needed? hadoop-3.2 profile should be enabled by default so `YarnClusterSuite` should also successfully finishes without `-Phadoop-3.2`. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Run `YarnClusterSuite` with both SBT and Maven without `-Phadoop-3.2` and it successfully finished. ``` build/sbt -Pyarn "yarn/testOnly org.apache.spark.deploy.yarn.YarnClusterSuite" ... [info] Run completed in 5 minutes, 38 seconds. [info] Total number of tests run: 27 [info] Suites: completed 1, aborted 0 [info] Tests: succeeded 27, failed 0, canceled 0, ignored 0, pending 0 [info] All tests passed. build/mvn -Pyarn -pl resource-managers/yarn test -Dtest=none -DwildcardSuites=org.apache.spark.deploy.yarn.YarnClusterSuite ... Run completed in 5 minutes, 49 seconds. Total number of tests run: 27 Suites: completed 2, aborted 0 Tests: succeeded 27, failed 0, canceled 0, ignored 0, pending 0 All tests passed. ``` Closes #33276 from sarutak/fix-bouncy-issue. Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org> |
||
---|---|---|
.. | ||
kubernetes | ||
mesos | ||
yarn |