ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[MINOR][DOCS][K8S] Use hadoop-aws 3.2.2 in K8s example

Browse source 
### What changes were proposed in this pull request?

This PR aims to update `Hadoop` dependency in K8S doc example.

### Why are the changes needed?

Apache Spark 3.2.0 is using Apache Hadoop 3.2.2 by default.

### Does this PR introduce _any_ user-facing change?

No. This is a doc-only change.

### How was this patch tested?

N/A

Closes #31628 from dongjoon-hyun/minor-doc.

Authored-by: Dongjoon Hyun <dhyun@apple.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
This commit is contained in:
Dongjoon Hyun 2021-02-24 11:34:29 +09:00 committed by HyukjinKwon
parent b5afff59fa
commit a6dcd5544d
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/running-on-kubernetes.md
View file

@ -212,7 +212,7 @@ A typical example of this using S3 is via passing the following options:
```
...
--packages com.amazonaws:aws-java-sdk:1.7.4,org.apache.hadoop:hadoop-aws:2.7.6
--packages org.apache.hadoop:hadoop-aws:3.2.2
--conf spark.kubernetes.file.upload.path=s3a://<s3-bucket>/path
--conf spark.hadoop.fs.s3a.access.key=...
--conf spark.hadoop.fs.s3a.impl=org.apache.hadoop.fs.s3a.S3AFileSystem