[SPARK-24793][K8S] Enhance spark-submit for app management
- supports `--kill` & `--status` flags.
- supports globs which is useful in general check this long standing [issue](https://github.com/kubernetes/kubernetes/issues/17144#issuecomment-272052461) for kubectl.
Manually against running apps. Example output:
Submission Id reported at launch time:
```
2019-01-20 23:47:56 INFO Client:58 - Waiting for application spark-pi with submissionId spark:spark-pi-1548020873671-driver to finish...
```
Killing the app:
```
./bin/spark-submit --kill spark:spark-pi-1548020873671-driver --master k8s://https://192.168.2.8:8443
2019-01-20 23:48:07 WARN Utils:70 - Your hostname, universe resolves to a loopback address: 127.0.0.1; using 192.168.2.8 instead (on interface wlp2s0)
2019-01-20 23:48:07 WARN Utils:70 - Set SPARK_LOCAL_IP if you need to bind to another address
```
App terminates with 143 (SIGTERM, since we have tiny this should lead to [graceful shutdown](https://cloud.google.com/solutions/best-practices-for-building-containers)):
```
2019-01-20 23:48:08 INFO LoggingPodStatusWatcherImpl:58 - State changed, new state:
pod name: spark-pi-1548020873671-driver
namespace: spark
labels: spark-app-selector -> spark-e4730c80e1014b72aa77915a2203ae05, spark-role -> driver
pod uid: 0ba9a794-1cfd-11e9-8215-a434d9270a65
creation time: 2019-01-20T21:47:55Z
service account name: spark-sa
volumes: spark-local-dir-1, spark-conf-volume, spark-sa-token-b7wcm
node name: minikube
start time: 2019-01-20T21:47:55Z
phase: Running
container status:
container name: spark-kubernetes-driver
container image: skonto/spark:k8s-3.0.0
container state: running
container started at: 2019-01-20T21:48:00Z
2019-01-20 23:48:09 INFO LoggingPodStatusWatcherImpl:58 - State changed, new state:
pod name: spark-pi-1548020873671-driver
namespace: spark
labels: spark-app-selector -> spark-e4730c80e1014b72aa77915a2203ae05, spark-role -> driver
pod uid: 0ba9a794-1cfd-11e9-8215-a434d9270a65
creation time: 2019-01-20T21:47:55Z
service account name: spark-sa
volumes: spark-local-dir-1, spark-conf-volume, spark-sa-token-b7wcm
node name: minikube
start time: 2019-01-20T21:47:55Z
phase: Failed
container status:
container name: spark-kubernetes-driver
container image: skonto/spark:k8s-3.0.0
container state: terminated
container started at: 2019-01-20T21:48:00Z
container finished at: 2019-01-20T21:48:08Z
exit code: 143
termination reason: Error
2019-01-20 23:48:09 INFO LoggingPodStatusWatcherImpl:58 - Container final statuses:
container name: spark-kubernetes-driver
container image: skonto/spark:k8s-3.0.0
container state: terminated
container started at: 2019-01-20T21:48:00Z
container finished at: 2019-01-20T21:48:08Z
exit code: 143
termination reason: Error
2019-01-20 23:48:09 INFO Client:58 - Application spark-pi with submissionId spark:spark-pi-1548020873671-driver finished.
2019-01-20 23:48:09 INFO ShutdownHookManager:58 - Shutdown hook called
2019-01-20 23:48:09 INFO ShutdownHookManager:58 - Deleting directory /tmp/spark-f114b2e0-5605-4083-9203-a4b1c1f6059e
```
Glob scenario:
```
./bin/spark-submit --status spark:spark-pi* --master k8s://https://192.168.2.8:8443
2019-01-20 22:27:44 WARN Utils:70 - Your hostname, universe resolves to a loopback address: 127.0.0.1; using 192.168.2.8 instead (on interface wlp2s0)
2019-01-20 22:27:44 WARN Utils:70 - Set SPARK_LOCAL_IP if you need to bind to another address
Application status (driver):
pod name: spark-pi-1547948600328-driver
namespace: spark
labels: spark-app-selector -> spark-f13f01702f0b4503975ce98252d59b94, spark-role -> driver
pod uid: c576e1c6-1c54-11e9-8215-a434d9270a65
creation time: 2019-01-20T01:43:22Z
service account name: spark-sa
volumes: spark-local-dir-1, spark-conf-volume, spark-sa-token-b7wcm
node name: minikube
start time: 2019-01-20T01:43:22Z
phase: Running
container status:
container name: spark-kubernetes-driver
container image: skonto/spark:k8s-3.0.0
container state: running
container started at: 2019-01-20T01:43:27Z
Application status (driver):
pod name: spark-pi-1547948792539-driver
namespace: spark
labels: spark-app-selector -> spark-006d252db9b24f25b5069df357c30264, spark-role -> driver
pod uid: 38375b4b-1c55-11e9-8215-a434d9270a65
creation time: 2019-01-20T01:46:35Z
service account name: spark-sa
volumes: spark-local-dir-1, spark-conf-volume, spark-sa-token-b7wcm
node name: minikube
start time: 2019-01-20T01:46:35Z
phase: Succeeded
container status:
container name: spark-kubernetes-driver
container image: skonto/spark:k8s-3.0.0
container state: terminated
container started at: 2019-01-20T01:46:39Z
container finished at: 2019-01-20T01:46:56Z
exit code: 0
termination reason: Completed
```
Closes #23599 from skonto/submit_ops_extension.
Authored-by: Stavros Kontopoulos <stavros.kontopoulos@lightbend.com>
Signed-off-by: Marcelo Vanzin <vanzin@cloudera.com>
05168e725d