[MINOR][K8S] Invalid property "spark.driver.pod.name" is referenced in docs.
## What changes were proposed in this pull request? "Running on Kubernetes" references `spark.driver.pod.name` few places, and it should be `spark.kubernetes.driver.pod.name`. ## How was this patch tested? See changes Closes #23133 from Leemoonsoo/fix-driver-pod-name-prop. Authored-by: Lee moon soo <moon@apache.org> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This commit is contained in:
parent
0f56977f8c
commit
eea4a0330b
|
@ -166,7 +166,7 @@ hostname via `spark.driver.host` and your spark driver's port to `spark.driver.p
|
|||
|
||||
### Client Mode Executor Pod Garbage Collection
|
||||
|
||||
If you run your Spark driver in a pod, it is highly recommended to set `spark.driver.pod.name` to the name of that pod.
|
||||
If you run your Spark driver in a pod, it is highly recommended to set `spark.kubernetes.driver.pod.name` to the name of that pod.
|
||||
When this property is set, the Spark scheduler will deploy the executor pods with an
|
||||
[OwnerReference](https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/), which in turn will
|
||||
ensure that once the driver pod is deleted from the cluster, all of the application's executor pods will also be deleted.
|
||||
|
@ -175,7 +175,7 @@ an OwnerReference pointing to that pod will be added to each executor pod's Owne
|
|||
setting the OwnerReference to a pod that is not actually that driver pod, or else the executors may be terminated
|
||||
prematurely when the wrong pod is deleted.
|
||||
|
||||
If your application is not running inside a pod, or if `spark.driver.pod.name` is not set when your application is
|
||||
If your application is not running inside a pod, or if `spark.kubernetes.driver.pod.name` is not set when your application is
|
||||
actually running in a pod, keep in mind that the executor pods may not be properly deleted from the cluster when the
|
||||
application exits. The Spark scheduler attempts to delete these pods, but if the network request to the API server fails
|
||||
for any reason, these pods will remain in the cluster. The executor processes should exit when they cannot reach the
|
||||
|
|
Loading…
Reference in a new issue