ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-17710][HOTFIX] Fix ClassCircularityError in ReplSuite tests in Maven build: use 'Class.forName' instead of 'Utils.classForName'

Browse source 
## What changes were proposed in this pull request?
Fix ClassCircularityError in ReplSuite tests when Spark is built by Maven build.

## How was this patch tested?
(1)
```
build/mvn -DskipTests -Phadoop-2.3 -Pyarn -Phive -Phive-thriftserver -Pkinesis-asl -Pmesos clean package
```
Then test:
```
build/mvn -Dtest=none -DwildcardSuites=org.apache.spark.repl.ReplSuite test
```
ReplSuite tests passed

(2)
Manual Tests against some Spark applications in Yarn client mode and Yarn cluster mode. Need to check if spark caller contexts are written into HDFS hdfs-audit.log and Yarn RM audit log successfully.

Author: Weiqing Yang <yangweiqing001@gmail.com>

Closes #15286 from Sherry302/SPARK-16757.
This commit is contained in:
Weiqing Yang 2016-09-28 20:20:03 -05:00 committed by Tom Graves
parent 7d09232028
commit 7dfad4b132
1 changed files with 4 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
core/src/main/scala/org/apache/spark/util/Utils.scala
View file

@ -2489,8 +2489,10 @@ private[spark] class CallerContext(
def setCurrentContext(): Boolean = {
var succeed = false
try {
val callerContext = Utils.classForName("org.apache.hadoop.ipc.CallerContext")
val Builder = Utils.classForName("org.apache.hadoop.ipc.CallerContext$Builder")
// scalastyle:off classforname
val callerContext = Class.forName("org.apache.hadoop.ipc.CallerContext")
val Builder = Class.forName("org.apache.hadoop.ipc.CallerContext$Builder")
// scalastyle:on classforname
val builderInst = Builder.getConstructor(classOf[String]).newInstance(context)
val hdfsContext = Builder.getMethod("build").invoke(builderInst)
callerContext.getMethod("setCurrent", callerContext).invoke(null, hdfsContext)