fd899d6331
### What changes were proposed in this pull request? Instead of using ZStd codec directly, we use Spark's CompressionCodec which wraps ZStd codec in a buffered stream to avoid overhead excessive of JNI call while trying to compress/decompress small amount of data. Also, by using Spark's CompressionCodec, we can easily to make it configurable in the future if it's needed. ### Why are the changes needed? Faster performance. ### Does this PR introduce any user-facing change? No. ### How was this patch tested? Existing tests. Closes #26235 from dbtsai/optimizeDeser. Lead-authored-by: DB Tsai <d_tsai@apple.com> Co-authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> |
||
---|---|---|
.. | ||
CoalescedRDDBenchmark-jdk11-results.txt | ||
CoalescedRDDBenchmark-results.txt | ||
KryoBenchmark-jdk11-results.txt | ||
KryoBenchmark-results.txt | ||
KryoSerializerBenchmark-jdk11-results.txt | ||
KryoSerializerBenchmark-results.txt | ||
MapStatusesSerDeserBenchmark-jdk11-results.txt | ||
MapStatusesSerDeserBenchmark-results.txt | ||
PropertiesCloneBenchmark-jdk11-results.txt | ||
PropertiesCloneBenchmark-results.txt | ||
XORShiftRandomBenchmark-jdk11-results.txt | ||
XORShiftRandomBenchmark-results.txt |