[SPARK-17058][BUILD] Add maven snapshots-and-staging profile to build/test against staging artifacts
## What changes were proposed in this pull request? Adds a `snapshots-and-staging profile` so that RCs of projects like Hadoop and HBase can be used in developer-only build and test runs. There's a comment above the profile telling people not to use this in production. There's no attempt to do the same for SBT, as Ivy is different. ## How was this patch tested? Tested by building against the Hadoop 2.7.3 RC 1 JARs without the profile (and without any local copy of the 2.7.3 artifacts), the build failed ``` mvn install -DskipTests -Pyarn,hadoop-2.7,hive -Dhadoop.version=2.7.3 ... [INFO] ------------------------------------------------------------------------ [INFO] Building Spark Project Launcher 2.1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.pom [WARNING] The POM for org.apache.hadoop:hadoop-client:jar:2.7.3 is missing, no dependency information available Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.jar [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Spark Project Parent POM ........................... SUCCESS [ 4.482 s] [INFO] Spark Project Tags ................................. SUCCESS [ 17.402 s] [INFO] Spark Project Sketch ............................... SUCCESS [ 11.252 s] [INFO] Spark Project Networking ........................... SUCCESS [ 13.458 s] [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 9.043 s] [INFO] Spark Project Unsafe ............................... SUCCESS [ 16.027 s] [INFO] Spark Project Launcher ............................. FAILURE [ 1.653 s] [INFO] Spark Project Core ................................. SKIPPED ... ``` With the profile, the build completed ``` mvn install -DskipTests -Pyarn,hadoop-2.7,hive,snapshots-and-staging -Dhadoop.version=2.7.3 ``` Author: Steve Loughran <stevel@apache.org> Closes #14646 from steveloughran/stevel/SPARK-17058-support-asf-snapshots.
This commit is contained in:
parent
3c24299b71
commit
37d95227a2
48
pom.xml
48
pom.xml
|
@ -2693,6 +2693,54 @@
|
|||
</build>
|
||||
</profile>
|
||||
|
||||
<!--
|
||||
This is a profile to enable the use of the ASF snapshot and staging repositories
|
||||
during a build. It is useful when testing againt nightly or RC releases of dependencies.
|
||||
It MUST NOT be used when building copies of Spark to use in production of for distribution,
|
||||
-->
|
||||
<profile>
|
||||
<id>snapshots-and-staging</id>
|
||||
<properties>
|
||||
<!-- override point for ASF staging/snapshot repos -->
|
||||
<asf.staging>https://repository.apache.org/content/groups/staging/</asf.staging>
|
||||
<asf.snapshots>https://repository.apache.org/content/repositories/snapshots/</asf.snapshots>
|
||||
</properties>
|
||||
|
||||
<pluginRepositories>
|
||||
<pluginRepository>
|
||||
<id>ASF Staging</id>
|
||||
<url>${asf.staging}</url>
|
||||
</pluginRepository>
|
||||
<pluginRepository>
|
||||
<id>ASF Snapshots</id>
|
||||
<url>${asf.snapshots}</url>
|
||||
<snapshots>
|
||||
<enabled>true</enabled>
|
||||
</snapshots>
|
||||
<releases>
|
||||
<enabled>false</enabled>
|
||||
</releases>
|
||||
</pluginRepository>
|
||||
|
||||
</pluginRepositories>
|
||||
<repositories>
|
||||
<repository>
|
||||
<id>ASF Staging</id>
|
||||
<url>${asf.staging}</url>
|
||||
</repository>
|
||||
<repository>
|
||||
<id>ASF Snapshots</id>
|
||||
<url>${asf.snapshots}</url>
|
||||
<snapshots>
|
||||
<enabled>true</enabled>
|
||||
</snapshots>
|
||||
<releases>
|
||||
<enabled>false</enabled>
|
||||
</releases>
|
||||
</repository>
|
||||
</repositories>
|
||||
</profile>
|
||||
|
||||
<!--
|
||||
These empty profiles are available in some sub-modules. Declare them here so that
|
||||
maven does not complain when they're provided on the command line for a sub-module
|
||||
|
|
Loading…
Reference in a new issue