[MINOR][K8S] Invalid property "spark.driver.pod.name" is referenced in docs.
## What changes were proposed in this pull request? "Running on Kubernetes" references `spark.driver.pod.name` few places, and it should be `spark.kubernetes.driver.pod.name`. ## How was this patch tested? See changes Closes #23133 from Leemoonsoo/fix-driver-pod-name-prop. Authored-by: Lee moon soo <moon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This commit is contained in:
parent
0f56977f8c
commit
eea4a0330b
|
@ -166,7 +166,7 @@ hostname via `spark.driver.host` and your spark driver's port to `spark.driver.p
|
||||||
|
|
||||||
### Client Mode Executor Pod Garbage Collection
|
### Client Mode Executor Pod Garbage Collection
|
||||||
|
|
||||||
If you run your Spark driver in a pod, it is highly recommended to set `spark.driver.pod.name` to the name of that pod.
|
If you run your Spark driver in a pod, it is highly recommended to set `spark.kubernetes.driver.pod.name` to the name of that pod.
|
||||||
When this property is set, the Spark scheduler will deploy the executor pods with an
|
When this property is set, the Spark scheduler will deploy the executor pods with an
|
||||||
[OwnerReference](https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/), which in turn will
|
[OwnerReference](https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/), which in turn will
|
||||||
ensure that once the driver pod is deleted from the cluster, all of the application's executor pods will also be deleted.
|
ensure that once the driver pod is deleted from the cluster, all of the application's executor pods will also be deleted.
|
||||||
|
@ -175,7 +175,7 @@ an OwnerReference pointing to that pod will be added to each executor pod's Owne
|
||||||
setting the OwnerReference to a pod that is not actually that driver pod, or else the executors may be terminated
|
setting the OwnerReference to a pod that is not actually that driver pod, or else the executors may be terminated
|
||||||
prematurely when the wrong pod is deleted.
|
prematurely when the wrong pod is deleted.
|
||||||
|
|
||||||
If your application is not running inside a pod, or if `spark.driver.pod.name` is not set when your application is
|
If your application is not running inside a pod, or if `spark.kubernetes.driver.pod.name` is not set when your application is
|
||||||
actually running in a pod, keep in mind that the executor pods may not be properly deleted from the cluster when the
|
actually running in a pod, keep in mind that the executor pods may not be properly deleted from the cluster when the
|
||||||
application exits. The Spark scheduler attempts to delete these pods, but if the network request to the API server fails
|
application exits. The Spark scheduler attempts to delete these pods, but if the network request to the API server fails
|
||||||
for any reason, these pods will remain in the cluster. The executor processes should exit when they cannot reach the
|
for any reason, these pods will remain in the cluster. The executor processes should exit when they cannot reach the
|
||||||
|
|
Loading…
Reference in a new issue