spark-instrumented-optimizer/core
Shixiong Zhu 66450bbc1b
[SPARK-26665][CORE] Fix a bug that BlockTransferService.fetchBlockSync may hang forever
## What changes were proposed in this pull request?

`ByteBuffer.allocate` may throw `OutOfMemoryError` when the block is large but no enough memory is available. However, when this happens, right now BlockTransferService.fetchBlockSync will just hang forever as its `BlockFetchingListener. onBlockFetchSuccess` doesn't complete `Promise`.

This PR catches `Throwable` and uses the error to complete `Promise`.

## How was this patch tested?

Added a unit test. Since I cannot make `ByteBuffer.allocate` throw `OutOfMemoryError`, I passed a negative size to make `ByteBuffer.allocate` fail. Although the error type is different, it should trigger the same code path.

Closes #23590 from zsxwing/SPARK-26665.

Authored-by: Shixiong Zhu <zsxwing@gmail.com>
Signed-off-by: Shixiong Zhu <zsxwing@gmail.com>
2019-01-22 09:00:52 -08:00
..
benchmarks [SPARK-25839][CORE] Implement use of KryoPool in KryoSerializer 2018-11-10 12:51:24 -06:00
src [SPARK-26665][CORE] Fix a bug that BlockTransferService.fetchBlockSync may hang forever 2019-01-22 09:00:52 -08:00
pom.xml [SPARK-22128][CORE][BUILD] Add paranamer dependency to core module 2019-01-10 00:40:21 -08:00