[SPARK-20940][CORE] Replace IllegalAccessError with IllegalStateException
## What changes were proposed in this pull request? `IllegalAccessError` is a fatal error (a subclass of LinkageError) and its meaning is `Thrown if an application attempts to access or modify a field, or to call a method that it does not have access to`. Throwing a fatal error for AccumulatorV2 is not necessary and is pretty bad because it usually will just kill executors or SparkContext ([SPARK-20666](https://issues.apache.org/jira/browse/SPARK-20666) is an example of killing SparkContext due to `IllegalAccessError`). I think the correct type of exception in AccumulatorV2 should be `IllegalStateException`. ## How was this patch tested? Jenkins Author: Shixiong Zhu <shixiong@databricks.com> Closes #18168 from zsxwing/SPARK-20940.
This commit is contained in:
parent
2bc3272880
commit
24db35826a
|
@ -68,7 +68,7 @@ abstract class AccumulatorV2[IN, OUT] extends Serializable {
|
|||
|
||||
private def assertMetadataNotNull(): Unit = {
|
||||
if (metadata == null) {
|
||||
throw new IllegalAccessError("The metadata of this accumulator has not been assigned yet.")
|
||||
throw new IllegalStateException("The metadata of this accumulator has not been assigned yet.")
|
||||
}
|
||||
}
|
||||
|
||||
|
@ -265,7 +265,7 @@ private[spark] object AccumulatorContext {
|
|||
// Since we are storing weak references, we must check whether the underlying data is valid.
|
||||
val acc = ref.get
|
||||
if (acc eq null) {
|
||||
throw new IllegalAccessError(s"Attempted to access garbage collected accumulator $id")
|
||||
throw new IllegalStateException(s"Attempted to access garbage collected accumulator $id")
|
||||
}
|
||||
acc
|
||||
}
|
||||
|
|
|
@ -210,7 +210,7 @@ class AccumulatorSuite extends SparkFunSuite with Matchers with LocalSparkContex
|
|||
assert(ref.get.isEmpty)
|
||||
|
||||
// Getting a garbage collected accum should throw error
|
||||
intercept[IllegalAccessError] {
|
||||
intercept[IllegalStateException] {
|
||||
AccumulatorContext.get(accId)
|
||||
}
|
||||
|
||||
|
|
Loading…
Reference in a new issue