[SPARK-33353][BUILD] Cache dependencies for Coursier with new sbt in GitHub Actions
### What changes were proposed in this pull request? This PR change the behavior of GitHub Actions job that caches dependencies. SPARK-33226 upgraded sbt to 1.4.1. As of 1.3.0, sbt uses Coursier as the dependency resolver / fetcher. So let's change the dependency cache configuration for the GitHub Actions job. ### Why are the changes needed? To make build faster with Coursier for the GitHub Actions job. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Should be done by GitHub Actions itself. Closes #30259 from sarutak/coursier-cache. Authored-by: Kousuke Saruta <sarutak@oss.nttdata.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
This commit is contained in:
parent
26ea417b14
commit
208b94e4c1
32
.github/workflows/build_and_test.yml
vendored
32
.github/workflows/build_and_test.yml
vendored
|
@ -111,13 +111,13 @@ jobs:
|
|||
key: ${{ matrix.java }}-${{ matrix.hadoop }}-maven-${{ hashFiles('**/pom.xml') }}
|
||||
restore-keys: |
|
||||
${{ matrix.java }}-${{ matrix.hadoop }}-maven-
|
||||
- name: Cache Ivy local repository
|
||||
- name: Cache Coursier local repository
|
||||
uses: actions/cache@v2
|
||||
with:
|
||||
path: ~/.ivy2/cache
|
||||
key: ${{ matrix.java }}-${{ matrix.hadoop }}-ivy-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
|
||||
path: ~/.cache/coursier
|
||||
key: ${{ matrix.java }}-${{ matrix.hadoop }}-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
|
||||
restore-keys: |
|
||||
${{ matrix.java }}-${{ matrix.hadoop }}-ivy-
|
||||
${{ matrix.java }}-${{ matrix.hadoop }}-coursier-
|
||||
- name: Install JDK ${{ matrix.java }}
|
||||
uses: actions/setup-java@v1
|
||||
with:
|
||||
|
@ -206,13 +206,13 @@ jobs:
|
|||
key: pyspark-maven-${{ hashFiles('**/pom.xml') }}
|
||||
restore-keys: |
|
||||
pyspark-maven-
|
||||
- name: Cache Ivy local repository
|
||||
- name: Cache Coursier local repository
|
||||
uses: actions/cache@v2
|
||||
with:
|
||||
path: ~/.ivy2/cache
|
||||
key: pyspark-ivy-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
|
||||
path: ~/.cache/coursier
|
||||
key: pyspark-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
|
||||
restore-keys: |
|
||||
pyspark-ivy-
|
||||
pyspark-coursier-
|
||||
- name: Install Python 3.6
|
||||
uses: actions/setup-python@v2
|
||||
with:
|
||||
|
@ -282,13 +282,13 @@ jobs:
|
|||
key: sparkr-maven-${{ hashFiles('**/pom.xml') }}
|
||||
restore-keys: |
|
||||
sparkr-maven-
|
||||
- name: Cache Ivy local repository
|
||||
- name: Cache Coursier local repository
|
||||
uses: actions/cache@v2
|
||||
with:
|
||||
path: ~/.ivy2/cache
|
||||
key: sparkr-ivy-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
|
||||
path: ~/.cache/coursier
|
||||
key: sparkr-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
|
||||
restore-keys: |
|
||||
sparkr-ivy-
|
||||
sparkr-coursier-
|
||||
- name: Run tests
|
||||
run: |
|
||||
mkdir -p ~/.m2
|
||||
|
@ -404,13 +404,13 @@ jobs:
|
|||
steps:
|
||||
- name: Checkout Spark repository
|
||||
uses: actions/checkout@v2
|
||||
- name: Cache Ivy local repository
|
||||
- name: Cache Coursier local repository
|
||||
uses: actions/cache@v2
|
||||
with:
|
||||
path: ~/.ivy2/cache
|
||||
key: scala-213-ivy-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
|
||||
path: ~/.cache/coursier
|
||||
key: scala-213-coursier-${{ hashFiles('**/pom.xml', '**/plugins.sbt') }}
|
||||
restore-keys: |
|
||||
scala-213-ivy-
|
||||
scala-213-coursier-
|
||||
- name: Install Java 11
|
||||
uses: actions/setup-java@v1
|
||||
with:
|
||||
|
|
Loading…
Reference in a new issue