ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity

[SPARK-31744][R][INFRA] Remove Hive dependency in AppVeyor build temporarily

Browse source 
### What changes were proposed in this pull request?

This PR targets to remove Hive profile in SparkR build at AppVeyor in order to:
- Speed up the build. Currently, SparkR build is [reaching the time limit](https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/32853533).
- Clean up the build profile.

### Why are the changes needed?

We're hitting a time limit issue again and this PR could reduce the build time.

Seems like we're [already skipping Hive related tests in SparkR](https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/32853533) for some reasons, see below:

```
test_sparkSQL.R:307: skip: create DataFrame from RDD
Reason: Hive is not build with SparkSQL, skipped
test_sparkSQL.R:1341: skip: test HiveContext
Reason: Hive is not build with SparkSQL, skipped
test_sparkSQL.R:2813: skip: read/write ORC files
Reason: Hive is not build with SparkSQL, skipped
test_sparkSQL.R:2834: skip: read/write ORC files - compression option
Reason: Hive is not build with SparkSQL, skipped
test_sparkSQL.R:3727: skip: enableHiveSupport on SparkSession
Reason: Hive is not build with SparkSQL, skipped
```

Although we build with Hive profile. So, the Hive profile is useless here.

### Does this PR introduce _any_ user-facing change?

No, dev-only.

### How was this patch tested?

AppVeyor will test it out.

Closes #28564 from HyukjinKwon/SPARK-31744.

Authored-by: HyukjinKwon <gurwls223@apache.org>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
This commit is contained in:
HyukjinKwon 2020-05-17 21:31:06 -07:00 committed by Dongjoon Hyun
parent 097d5098cc
commit f352cef077
No known key found for this signature in database
GPG key ID: EDA00CE834F0FC5C
1 changed files with 2 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
appveyor.yml
View file

@ -48,7 +48,8 @@ install:
build_script: build_script:
# '-Djna.nosys=true' is required to avoid kernel32.dll load failure. # '-Djna.nosys=true' is required to avoid kernel32.dll load failure.
# See SPARK-28759. # See SPARK-28759.
- cmd: mvn -DskipTests -Psparkr -Phive -Djna.nosys=true package # Ideally we should check the tests related to Hive in SparkR as well (SPARK-31745).
- cmd: mvn -DskipTests -Psparkr -Djna.nosys=true package
environment: environment:
NOT_CRAN: true NOT_CRAN: true