Matthew Jones 2fff060b6d [SPARK-36009][GRAPHX] Add missing GraphX classes to registerKryoClasses util method ... ### What changes were proposed in this pull request? Add `ShippableVertexPartition` and `RoutingTablePartition` to the classes registered with Kryo in `GraphXUtils.registerKryoClasses`. ### Why are the changes needed? `VertexRDDImpl` uses an `RDD[ShippableVertexPartition[VD]]` however, `GraphXUtils.registerKryoClasses` does not register `ShippableVertexPartition`. This means when running with `spark.kryo.registrationRequired` set to `true`, we get a "Class is not registered" exception. This is an issue as it prevents other unregistered classes from being discovered using `spark.kryo.registrationRequired` as the first unregistered class found halts the whole job. It also potentially decreases the serialised size of the RDD when using Kryo. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Tested manually by running with `spark.kryo.registrationRequired` set to `true` and verifying no "Class is not registered" exception was thrown. Closes #32973 from matthewrj/bug/register-graphx-classes. Lead-authored-by: Matthew Jones <matthew@vitaler.com> Co-authored-by: Matthew Jones <mlety2@gmail.com> Signed-off-by: Sean Owen <srowen@gmail.com>		2021-07-06 07:25:22 -05:00
..
src	[SPARK-36009][GRAPHX] Add missing GraphX classes to registerKryoClasses util method	2021-07-06 07:25:22 -05:00
pom.xml	[SPARK-35996][BUILD] Setting version to 3.3.0-SNAPSHOT	2021-07-02 13:47:36 -07:00