ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-33046][DOCS] Update how to build doc for Scala 2.13 with sbt

Browse source 
### What changes were proposed in this pull request?

This PR fixes the description how to build Spark for Scala 2.13 with sbt.
In the current doc, how to build Spark for Scala 2.13 with sbt is described like:
![scala-2 13-build-before](https://user-images.githubusercontent.com/4736016/94816248-80c3e900-0436-11eb-9bc2-99af5786971a.png)

But build fails with this command because scala-2.13 profile is not enabled and scala-parallel-collections is absent.

```
[error] /home/kou/work/oss/spark-scala-2.13/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala:23: object parallel is not a member of package collection
```

The correct command should be:
```
build/sbt -Pspark-2.13 compile
```

### Why are the changes needed?

The build command is wrong.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

I checked that `sbt -Pspark-2.13` is correct with the following command:
```
build/sbt -Dscala.version=2.13.3 -Phive -Phive-thriftserver -Pyarn -Pkubernetes  compile
```

I also build the modified doc and checked the generated html:
![spark-scala-2 13-build-doc-after](https://user-images.githubusercontent.com/4736016/94869259-f2745500-047f-11eb-89e5-20816f3ed24d.png)

Closes #29921 from sarutak/fix-scala-2.13-build-doc.

Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com>
Signed-off-by: Sean Owen <srowen@gmail.com>
This commit is contained in:
Kousuke Saruta 2020-10-01 18:01:23 -05:00 committed by Sean Owen
parent e62d24717e
commit 005999721f
1 changed files with 4 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
docs/building-spark.md
View file

@ -265,15 +265,13 @@ Change the major Scala version using (e.g. 2.13):
./dev/change-scala-version.sh 2.13
For Maven, please enable the profile (e.g. 2.13):
Enable the profile (e.g. 2.13):
# For Maven
./build/mvn -Pscala-2.13 compile
For SBT, specify a complete scala version using (e.g. 2.13.0):
./build/sbt -Dscala.version=2.13.0
Otherwise, the sbt-pom-reader plugin will use the `scala.version` specified in the spark-parent pom.
# For sbt
./build/sbt -Pscala-2.13 compile
## Running Jenkins tests with Github Enterprise