[SPARK-8819] Fix build for maven 3.3.x
This is a workaround for MSHADE-148, which leads to an infinite loop when building Spark with maven 3.3.x. This was originally caused by #6441, which added a bunch of test dependencies on the spark-core test module. Recently, it was revealed by #7193. This patch adds a `-Prelease` profile. If present, it will set `createDependencyReducedPom` to true. The consequences are: - If you are releasing Spark with this profile, you are fine as long as you use maven 3.2.x or before. - If you are releasing Spark without this profile, you will run into SPARK-8781. - If you are not releasing Spark but you are using this profile, you may run into SPARK-8819. - If you are not releasing Spark and you did not include this profile, you are fine. This is all documented in `pom.xml` and tested locally with both versions of maven. Author: Andrew Or <andrew@databricks.com> Closes #7219 from andrewor14/fix-maven-build and squashes the following commits: 1d37e87 [Andrew Or] Merge branch 'master' of github.com:apache/spark into fix-maven-build 3574ae4 [Andrew Or] Review comments f39199c [Andrew Or] Create a -Prelease profile that flags `createDependencyReducedPom`
This commit is contained in:
parent
d4d6d31db5
commit
9eae5fa642
|
@ -118,13 +118,13 @@ if [[ ! "$@" =~ --skip-publish ]]; then
|
|||
|
||||
rm -rf $SPARK_REPO
|
||||
|
||||
build/mvn -DskipTests -Pyarn -Phive \
|
||||
build/mvn -DskipTests -Pyarn -Phive -Prelease-profile\
|
||||
-Phive-thriftserver -Phadoop-2.2 -Pspark-ganglia-lgpl -Pkinesis-asl \
|
||||
clean install
|
||||
|
||||
./dev/change-version-to-2.11.sh
|
||||
|
||||
build/mvn -DskipTests -Pyarn -Phive \
|
||||
build/mvn -DskipTests -Pyarn -Phive -Prelease-profile\
|
||||
-Dscala-2.11 -Phadoop-2.2 -Pspark-ganglia-lgpl -Pkinesis-asl \
|
||||
clean install
|
||||
|
||||
|
|
24
pom.xml
24
pom.xml
|
@ -161,6 +161,8 @@
|
|||
<fasterxml.jackson.version>2.4.4</fasterxml.jackson.version>
|
||||
<snappy.version>1.1.1.7</snappy.version>
|
||||
<netlib.java.version>1.1.2</netlib.java.version>
|
||||
<!-- For maven shade plugin (see SPARK-8819) -->
|
||||
<create.dependency.reduced.pom>false</create.dependency.reduced.pom>
|
||||
|
||||
<test.java.home>${java.home}</test.java.home>
|
||||
|
||||
|
@ -1440,6 +1442,8 @@
|
|||
<version>2.3</version>
|
||||
<configuration>
|
||||
<shadedArtifactAttached>false</shadedArtifactAttached>
|
||||
<!-- Work around MSHADE-148. See SPARK-8819. -->
|
||||
<createDependencyReducedPom>${create.dependency.reduced.pom}</createDependencyReducedPom>
|
||||
<artifactSet>
|
||||
<includes>
|
||||
<!-- At a minimum we must include this to force effective pom generation -->
|
||||
|
@ -1826,6 +1830,26 @@
|
|||
</properties>
|
||||
</profile>
|
||||
|
||||
<profile>
|
||||
<!--
|
||||
Use this profile only for making Spark releases. Note that due to SPARK-8819,
|
||||
you must use maven version 3.2.x or before to avoid running into MSHADE-148.
|
||||
-->
|
||||
<id>release-profile</id>
|
||||
<properties>
|
||||
<!--
|
||||
The maven shade plugin has a bug where enabling the `createDependencyReducedPom`
|
||||
property causes maven to go into an infinite loop (MSHADE-148). This is only an
|
||||
issue for the Spark build if the maven version is 3.3.x or newer (SPARK-8819).
|
||||
|
||||
However, since disabling this property has the side effect of not resolving
|
||||
variables in the released pom files (SPARK-8781), we need to enable this during
|
||||
releases.
|
||||
-->
|
||||
<create.dependency.reduced.pom>true</create.dependency.reduced.pom>
|
||||
</properties>
|
||||
</profile>
|
||||
|
||||
<!--
|
||||
These empty profiles are available in some sub-modules. Declare them here so that
|
||||
maven does not complain when they're provided on the command line for a sub-module
|
||||
|
|
Loading…
Reference in a new issue