6cd23482d1
## What changes were proposed in this pull request? * Implement (optional) use of KryoPool in KryoSerializer, an alternative to the existing implementation of caching a Kryo instance inside KryoSerializerInstance * Add config key & documentation of spark.kryo.pool in order to turn this on * Add benchmark KryoSerializerBenchmark to compare new and old implementation * Add results of benchmark ## How was this patch tested? Added new tests inside KryoSerializerSuite to test the pool implementation as well as added the pool option to the existing regression testing for SPARK-7766 This is my original work and I license the work to the project under the project’s open source license. Closes #22855 from patrickbrownsync/kryo-pool. Authored-by: Patrick Brown <patrick.brown@blyncsy.com> Signed-off-by: Sean Owen <sean.owen@databricks.com>
13 lines
725 B
Plaintext
13 lines
725 B
Plaintext
================================================================================================
|
|
Benchmark KryoPool vs "pool of 1"
|
|
================================================================================================
|
|
|
|
Java HotSpot(TM) 64-Bit Server VM 1.8.0_131-b11 on Mac OS X 10.14
|
|
Intel(R) Core(TM) i7-4770HQ CPU @ 2.20GHz
|
|
Benchmark KryoPool vs "pool of 1": Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
|
|
------------------------------------------------------------------------------------------------
|
|
KryoPool:true 2682 / 3425 0.0 5364627.9 1.0X
|
|
KryoPool:false 8176 / 9292 0.0 16351252.2 0.3X
|
|
|
|
|