99d2e4e007
When replicating large cached RDD blocks, it can be helpful to replicate them as a stream, to avoid using large amounts of memory during the transfer. This also allows blocks larger than 2GB to be replicated. Added unit tests in DistributedSuite. Also ran tests on a cluster for blocks > 2gb. Closes #21451 from squito/clean_replication. Authored-by: Imran Rashid <irashid@cloudera.com> Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com> |
||
|---|---|---|
| .. | ||
| build.properties | ||
| MimaBuild.scala | ||
| MimaExcludes.scala | ||
| plugins.sbt | ||
| SparkBuild.scala | ||