266b92faff
## What changes were proposed in this pull request?
New config var: spark.mesos.docker.containerizer={"mesos","docker" (default)}
This adds support for running docker containers via the Mesos unified containerizer: http://mesos.apache.org/documentation/latest/container-image/
The benefit is losing the dependency on `dockerd`, and all the costs which it incurs.
I've also updated the supported Mesos version to 0.28.2 for support of the required protobufs.
This is blocked on: https://github.com/apache/spark/pull/14167
## How was this patch tested?
- manually testing jobs submitted with both "mesos" and "docker" settings for the new config var.
- spark/mesos integration test suite
Author: Michael Gummelt <mgummelt@mesosphere.io>
Closes #14275 from mgummelt/unified-containerizer.
24 lines
615 B
YAML
24 lines
615 B
YAML
highlighter: pygments
|
|
markdown: kramdown
|
|
gems:
|
|
- jekyll-redirect-from
|
|
|
|
# For some reason kramdown seems to behave differently on different
|
|
# OS/packages wrt encoding. So we hard code this config.
|
|
kramdown:
|
|
entity_output: numeric
|
|
|
|
include:
|
|
- _static
|
|
- _modules
|
|
|
|
# These allow the documentation to be updated with newer releases
|
|
# of Spark, Scala, and Mesos.
|
|
SPARK_VERSION: 2.1.0-SNAPSHOT
|
|
SPARK_VERSION_SHORT: 2.1.0
|
|
SCALA_BINARY_VERSION: "2.11"
|
|
SCALA_VERSION: "2.11.7"
|
|
MESOS_VERSION: 1.0.0
|
|
SPARK_ISSUE_TRACKER_URL: https://issues.apache.org/jira/browse/SPARK
|
|
SPARK_GITHUB_URL: https://github.com/apache/spark
|