[SPARK-14911] [CORE] Fix a potential data race in TaskMemoryManager
## What changes were proposed in this pull request?
[[SPARK-13210][SQL] catch OOM when allocate memory and expand array](37bc203c8d
) introduced an `acquiredButNotUsed` field, but it might not be correctly synchronized:
- the write `acquiredButNotUsed += acquired` is guarded by `this` lock (see [here](https://github.com/apache/spark/blame/master/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java#L271));
- the read `memoryManager.releaseExecutionMemory(acquiredButNotUsed, taskAttemptId, tungstenMemoryMode)` (see [here](https://github.com/apache/spark/blame/master/core/src/main/java/org/apache/spark/memory/TaskMemoryManager.java#L400)) might not be correctly synchronized, and thus might not see `acquiredButNotUsed`'s most recent value.
This patch makes `acquiredButNotUsed` volatile to fix this.
## How was this patch tested?
This should be covered by existing suits.
Author: Liwei Lin <lwlin7@gmail.com>
Closes #12681 from lw-lin/fix-acquiredButNotUsed.
This commit is contained in:
parent
8fda5a73dc
commit
b2a4560648
|
@ -114,7 +114,7 @@ public class TaskMemoryManager {
|
|||
/**
|
||||
* The amount of memory that is acquired but not used.
|
||||
*/
|
||||
private long acquiredButNotUsed = 0L;
|
||||
private volatile long acquiredButNotUsed = 0L;
|
||||
|
||||
/**
|
||||
* Construct a new TaskMemoryManager.
|
||||
|
|
Loading…
Reference in a new issue