ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
spark-instrumented-optimizer/common
History
Sean Owen 043dd531a0 [SPARK-36704][CORE] Expand exception handling to more Java 9 cases where reflection is limited at runtime, when reflecting to manage DirectByteBuffer settings 
### What changes were proposed in this pull request?

Improve exception handling in the Platform initialization, where it attempts to assess whether reflection is possible to modify DirectByteBuffer. This can apparently fail in more cases on Java 9+ than are currently handled, whereas Spark can continue without reflection if needed.

More detailed comments on the change inline.

### Why are the changes needed?

This exception seems to be possible and fails startup:

```
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make private java.nio.DirectByteBuffer(long,int) accessible: module java.base does not "opens java.nio" to unnamed module 71e9ddb4
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:357)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
        at java.base/java.lang.reflect.Constructor.checkCanSetAccessible(Constructor.java:188)
        at java.base/java.lang.reflect.Constructor.setAccessible(Constructor.java:181)
        at org.apache.spark.unsafe.Platform.<clinit>(Platform.java:56)
```

### Does this PR introduce _any_ user-facing change?

Should strictly allow Spark to continue in more cases.

### How was this patch tested?

Existing tests.

Closes #33947 from srowen/SPARK-36704.

Authored-by: Sean Owen <srowen@gmail.com>
Signed-off-by: Sean Owen <srowen@gmail.com>
(cherry picked from commit e5283f5ed5)
Signed-off-by: Sean Owen <srowen@gmail.com>
2021-09-11 13:38:20 -05:00
..
kvstore Preparing development version 3.2.1-SNAPSHOT 2021-08-31 17:04:14 +00:00
network-common Preparing development version 3.2.1-SNAPSHOT 2021-08-31 17:04:14 +00:00
network-shuffle Preparing development version 3.2.1-SNAPSHOT 2021-08-31 17:04:14 +00:00
network-yarn Preparing development version 3.2.1-SNAPSHOT 2021-08-31 17:04:14 +00:00
sketch Preparing development version 3.2.1-SNAPSHOT 2021-08-31 17:04:14 +00:00
tags Preparing development version 3.2.1-SNAPSHOT 2021-08-31 17:04:14 +00:00
unsafe [SPARK-36704][CORE] Expand exception handling to more Java 9 cases where reflection is limited at runtime, when reflecting to manage DirectByteBuffer settings 2021-09-11 13:38:20 -05:00