de59e01aa4
Kubernetes supports marking secrets and config maps as immutable to gain performance. https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable For K8s clusters that run many thousands of Spark applications, this can yield significant reduction in load on the kube-apiserver. From the K8s docs: > For clusters that extensively use Secrets (at least tens of thousands of unique Secret to Pod mounts), preventing changes to their data has the following advantages: > - protects you from accidental (or unwanted) updates that could cause applications outages > - improves performance of your cluster by significantly reducing load on kube-apiserver, by closing watches for secrets marked as immutable. For any secrets and config maps we create in Spark that are immutable, we could mark them as immutable by including the following when building the secret/config map ``` .withImmutable(true) ``` This feature has been supported in K8s as beta since K8s 1.19 and as GA since K8s 1.21 ### What changes were proposed in this pull request? All K8s secrets and config maps created by Spark are marked "immutable". ### Why are the changes needed? See description above. ### Does this PR introduce _any_ user-facing change? Don't think so ### How was this patch tested? Augmented existing unit tests. Closes #32588 from ashrayjain/patch-1. Authored-by: Ashray Jain <ashrayjain@users.noreply.github.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> |
||
|---|---|---|
| .. | ||
| kubernetes | ||
| mesos | ||
| yarn | ||