[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
HikariCP/2.5.1//HikariCP-2.5.1.jar
|
|
|
|
|
JLargeArrays/1.5//JLargeArrays-1.5.jar
|
|
|
|
|
JTransforms/3.1//JTransforms-3.1.jar
|
2020-07-25 13:58:25 -04:00
|
|
|
RoaringBitmap/0.9.0//RoaringBitmap-0.9.0.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
ST4/4.0.4//ST4-4.0.4.jar
|
|
|
|
|
activation/1.1.1//activation-1.1.1.jar
|
|
|
|
|
aircompressor/0.10//aircompressor-0.10.jar
|
|
|
|
|
algebra_2.12/2.0.0-M2//algebra_2.12-2.0.0-M2.jar
|
|
|
|
|
antlr-runtime/3.5.2//antlr-runtime-3.5.2.jar
|
|
|
|
|
antlr4-runtime/4.7.1//antlr4-runtime-4.7.1.jar
|
|
|
|
|
aopalliance-repackaged/2.6.1//aopalliance-repackaged-2.6.1.jar
|
|
|
|
|
aopalliance/1.0//aopalliance-1.0.jar
|
|
|
|
|
apacheds-i18n/2.0.0-M15//apacheds-i18n-2.0.0-M15.jar
|
|
|
|
|
apacheds-kerberos-codec/2.0.0-M15//apacheds-kerberos-codec-2.0.0-M15.jar
|
|
|
|
|
api-asn1-api/1.0.0-M20//api-asn1-api-1.0.0-M20.jar
|
|
|
|
|
api-util/1.0.0-M20//api-util-1.0.0-M20.jar
|
|
|
|
|
arpack_combined_all/0.1//arpack_combined_all-0.1.jar
|
2020-11-09 22:07:16 -05:00
|
|
|
arrow-format/2.0.0//arrow-format-2.0.0.jar
|
|
|
|
|
arrow-memory-core/2.0.0//arrow-memory-core-2.0.0.jar
|
|
|
|
|
arrow-memory-netty/2.0.0//arrow-memory-netty-2.0.0.jar
|
|
|
|
|
arrow-vector/2.0.0//arrow-vector-2.0.0.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
audience-annotations/0.5.0//audience-annotations-0.5.0.jar
|
|
|
|
|
automaton/1.11-8//automaton-1.11-8.jar
|
|
|
|
|
avro-ipc/1.8.2//avro-ipc-1.8.2.jar
|
|
|
|
|
avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar
|
|
|
|
|
avro/1.8.2//avro-1.8.2.jar
|
|
|
|
|
bonecp/0.8.0.RELEASE//bonecp-0.8.0.RELEASE.jar
|
|
|
|
|
breeze-macros_2.12/1.0//breeze-macros_2.12-1.0.jar
|
|
|
|
|
breeze_2.12/1.0//breeze_2.12-1.0.jar
|
|
|
|
|
cats-kernel_2.12/2.0.0-M4//cats-kernel_2.12-2.0.0-M4.jar
|
2020-01-19 21:39:38 -05:00
|
|
|
chill-java/0.9.5//chill-java-0.9.5.jar
|
|
|
|
|
chill_2.12/0.9.5//chill_2.12-0.9.5.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
commons-beanutils/1.9.4//commons-beanutils-1.9.4.jar
|
|
|
|
|
commons-cli/1.2//commons-cli-1.2.jar
|
|
|
|
|
commons-codec/1.10//commons-codec-1.10.jar
|
|
|
|
|
commons-collections/3.2.2//commons-collections-3.2.2.jar
|
2020-08-20 16:26:39 -04:00
|
|
|
commons-compiler/3.0.16//commons-compiler-3.0.16.jar
|
2020-11-09 21:08:55 -05:00
|
|
|
commons-compress/1.20//commons-compress-1.20.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
commons-configuration/1.6//commons-configuration-1.6.jar
|
2020-11-09 17:33:27 -05:00
|
|
|
commons-crypto/1.1.0//commons-crypto-1.1.0.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
commons-dbcp/1.4//commons-dbcp-1.4.jar
|
|
|
|
|
commons-digester/1.8//commons-digester-1.8.jar
|
|
|
|
|
commons-httpclient/3.1//commons-httpclient-3.1.jar
|
|
|
|
|
commons-io/2.4//commons-io-2.4.jar
|
|
|
|
|
commons-lang/2.6//commons-lang-2.6.jar
|
2020-06-23 03:59:55 -04:00
|
|
|
commons-lang3/3.10//commons-lang3-3.10.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
commons-logging/1.1.3//commons-logging-1.1.3.jar
|
|
|
|
|
commons-math3/3.4.1//commons-math3-3.4.1.jar
|
|
|
|
|
commons-net/3.1//commons-net-3.1.jar
|
|
|
|
|
commons-pool/1.5.4//commons-pool-1.5.4.jar
|
|
|
|
|
commons-text/1.6//commons-text-1.6.jar
|
|
|
|
|
compress-lzf/1.0.3//compress-lzf-1.0.3.jar
|
|
|
|
|
core/1.1.2//core-1.1.2.jar
|
|
|
|
|
curator-client/2.7.1//curator-client-2.7.1.jar
|
|
|
|
|
curator-framework/2.7.1//curator-framework-2.7.1.jar
|
|
|
|
|
curator-recipes/2.7.1//curator-recipes-2.7.1.jar
|
|
|
|
|
datanucleus-api-jdo/4.2.4//datanucleus-api-jdo-4.2.4.jar
|
|
|
|
|
datanucleus-core/4.1.17//datanucleus-core-4.1.17.jar
|
|
|
|
|
datanucleus-rdbms/4.1.19//datanucleus-rdbms-4.1.19.jar
|
|
|
|
|
derby/10.12.1.1//derby-10.12.1.1.jar
|
|
|
|
|
dropwizard-metrics-hadoop-metrics2-reporter/0.1.2//dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar
|
|
|
|
|
flatbuffers-java/1.9.0//flatbuffers-java-1.9.0.jar
|
|
|
|
|
generex/1.0.2//generex-1.0.2.jar
|
|
|
|
|
gson/2.2.4//gson-2.2.4.jar
|
|
|
|
|
guava/14.0.1//guava-14.0.1.jar
|
|
|
|
|
guice-servlet/3.0//guice-servlet-3.0.jar
|
|
|
|
|
guice/3.0//guice-3.0.jar
|
|
|
|
|
hadoop-annotations/2.7.4//hadoop-annotations-2.7.4.jar
|
|
|
|
|
hadoop-auth/2.7.4//hadoop-auth-2.7.4.jar
|
|
|
|
|
hadoop-client/2.7.4//hadoop-client-2.7.4.jar
|
|
|
|
|
hadoop-common/2.7.4//hadoop-common-2.7.4.jar
|
|
|
|
|
hadoop-hdfs/2.7.4//hadoop-hdfs-2.7.4.jar
|
|
|
|
|
hadoop-mapreduce-client-app/2.7.4//hadoop-mapreduce-client-app-2.7.4.jar
|
|
|
|
|
hadoop-mapreduce-client-common/2.7.4//hadoop-mapreduce-client-common-2.7.4.jar
|
|
|
|
|
hadoop-mapreduce-client-core/2.7.4//hadoop-mapreduce-client-core-2.7.4.jar
|
|
|
|
|
hadoop-mapreduce-client-jobclient/2.7.4//hadoop-mapreduce-client-jobclient-2.7.4.jar
|
|
|
|
|
hadoop-mapreduce-client-shuffle/2.7.4//hadoop-mapreduce-client-shuffle-2.7.4.jar
|
|
|
|
|
hadoop-yarn-api/2.7.4//hadoop-yarn-api-2.7.4.jar
|
|
|
|
|
hadoop-yarn-client/2.7.4//hadoop-yarn-client-2.7.4.jar
|
|
|
|
|
hadoop-yarn-common/2.7.4//hadoop-yarn-common-2.7.4.jar
|
|
|
|
|
hadoop-yarn-server-common/2.7.4//hadoop-yarn-server-common-2.7.4.jar
|
|
|
|
|
hadoop-yarn-server-web-proxy/2.7.4//hadoop-yarn-server-web-proxy-2.7.4.jar
|
2020-04-20 16:38:24 -04:00
|
|
|
hive-beeline/2.3.7//hive-beeline-2.3.7.jar
|
|
|
|
|
hive-cli/2.3.7//hive-cli-2.3.7.jar
|
|
|
|
|
hive-common/2.3.7//hive-common-2.3.7.jar
|
|
|
|
|
hive-exec/2.3.7/core/hive-exec-2.3.7-core.jar
|
|
|
|
|
hive-jdbc/2.3.7//hive-jdbc-2.3.7.jar
|
|
|
|
|
hive-llap-common/2.3.7//hive-llap-common-2.3.7.jar
|
|
|
|
|
hive-metastore/2.3.7//hive-metastore-2.3.7.jar
|
|
|
|
|
hive-serde/2.3.7//hive-serde-2.3.7.jar
|
2020-10-16 12:37:54 -04:00
|
|
|
hive-service-rpc/2.3.7//hive-service-rpc-2.3.7.jar
|
2020-04-20 16:38:24 -04:00
|
|
|
hive-shims-0.23/2.3.7//hive-shims-0.23-2.3.7.jar
|
|
|
|
|
hive-shims-common/2.3.7//hive-shims-common-2.3.7.jar
|
|
|
|
|
hive-shims-scheduler/2.3.7//hive-shims-scheduler-2.3.7.jar
|
|
|
|
|
hive-shims/2.3.7//hive-shims-2.3.7.jar
|
2020-10-01 15:41:40 -04:00
|
|
|
hive-storage-api/2.7.2//hive-storage-api-2.7.2.jar
|
2020-04-20 16:38:24 -04:00
|
|
|
hive-vector-code-gen/2.3.7//hive-vector-code-gen-2.3.7.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
hk2-api/2.6.1//hk2-api-2.6.1.jar
|
|
|
|
|
hk2-locator/2.6.1//hk2-locator-2.6.1.jar
|
|
|
|
|
hk2-utils/2.6.1//hk2-utils-2.6.1.jar
|
|
|
|
|
htrace-core/3.1.0-incubating//htrace-core-3.1.0-incubating.jar
|
|
|
|
|
httpclient/4.5.6//httpclient-4.5.6.jar
|
|
|
|
|
httpcore/4.4.12//httpcore-4.4.12.jar
|
|
|
|
|
istack-commons-runtime/3.0.8//istack-commons-runtime-3.0.8.jar
|
|
|
|
|
ivy/2.4.0//ivy-2.4.0.jar
|
|
|
|
|
jackson-annotations/2.10.0//jackson-annotations-2.10.0.jar
|
|
|
|
|
jackson-core-asl/1.9.13//jackson-core-asl-1.9.13.jar
|
|
|
|
|
jackson-core/2.10.0//jackson-core-2.10.0.jar
|
|
|
|
|
jackson-databind/2.10.0//jackson-databind-2.10.0.jar
|
|
|
|
|
jackson-dataformat-yaml/2.10.0//jackson-dataformat-yaml-2.10.0.jar
|
2020-11-03 01:23:26 -05:00
|
|
|
jackson-datatype-jsr310/2.11.2//jackson-datatype-jsr310-2.11.2.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
jackson-jaxrs/1.9.13//jackson-jaxrs-1.9.13.jar
|
|
|
|
|
jackson-mapper-asl/1.9.13//jackson-mapper-asl-1.9.13.jar
|
|
|
|
|
jackson-module-jaxb-annotations/2.10.0//jackson-module-jaxb-annotations-2.10.0.jar
|
|
|
|
|
jackson-module-paranamer/2.10.0//jackson-module-paranamer-2.10.0.jar
|
|
|
|
|
jackson-module-scala_2.12/2.10.0//jackson-module-scala_2.12-2.10.0.jar
|
|
|
|
|
jackson-xc/1.9.13//jackson-xc-1.9.13.jar
|
|
|
|
|
jakarta.activation-api/1.2.1//jakarta.activation-api-1.2.1.jar
|
|
|
|
|
jakarta.annotation-api/1.3.5//jakarta.annotation-api-1.3.5.jar
|
|
|
|
|
jakarta.inject/2.6.1//jakarta.inject-2.6.1.jar
|
|
|
|
|
jakarta.validation-api/2.0.2//jakarta.validation-api-2.0.2.jar
|
|
|
|
|
jakarta.ws.rs-api/2.1.6//jakarta.ws.rs-api-2.1.6.jar
|
|
|
|
|
jakarta.xml.bind-api/2.3.2//jakarta.xml.bind-api-2.3.2.jar
|
2020-08-20 16:26:39 -04:00
|
|
|
janino/3.0.16//janino-3.0.16.jar
|
2020-01-25 18:41:55 -05:00
|
|
|
javassist/3.25.0-GA//javassist-3.25.0-GA.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
javax.inject/1//javax.inject-1.jar
|
|
|
|
|
javax.jdo/3.2.0-m3//javax.jdo-3.2.0-m3.jar
|
|
|
|
|
javax.servlet-api/3.1.0//javax.servlet-api-3.1.0.jar
|
|
|
|
|
javolution/5.5.1//javolution-5.5.1.jar
|
[SPARK-33212][BUILD] Move to shaded clients for Hadoop 3.x profile
### What changes were proposed in this pull request?
This switches Spark to use shaded Hadoop clients, namely hadoop-client-api and hadoop-client-runtime, for Hadoop 3.x. For Hadoop 2.7, we'll still use the same modules such as hadoop-client.
In order to still keep default Hadoop profile to be hadoop-3.2, this defines the following Maven properties:
```
hadoop-client-api.artifact
hadoop-client-runtime.artifact
hadoop-client-minicluster.artifact
```
which default to:
```
hadoop-client-api
hadoop-client-runtime
hadoop-client-minicluster
```
but all switch to `hadoop-client` when the Hadoop profile is hadoop-2.7. A side affect from this is we'll import the same dependency multiple times. For this I have to disable Maven enforcer `banDuplicatePomDependencyVersions`.
Besides above, there are the following changes:
- explicitly add a few dependencies which are imported via transitive dependencies from Hadoop jars, but are removed from the shaded client jars.
- removed the use of `ProxyUriUtils.getPath` from `ApplicationMaster` which is a server-side/private API.
- modified `IsolatedClientLoader` to exclude `hadoop-auth` jars when Hadoop version is 3.x. This change should only matter when we're not sharing Hadoop classes with Spark (which is _mostly_ used in tests).
### Why are the changes needed?
This serves two purposes:
- to unblock Spark from upgrading to Hadoop 3.2.2/3.3.0+. Latest Hadoop versions have upgraded to use Guava 27+ and in order to adopt the latest Hadoop versions in Spark, we'll need to resolve the Guava conflicts. This takes the approach by switching to shaded client jars provided by Hadoop.
- avoid pulling 3rd party dependencies from Hadoop and avoid potential future conflicts.
### Does this PR introduce _any_ user-facing change?
When people use Spark with `hadoop-provided` option, they should make sure class path contains `hadoop-client-api` and `hadoop-client-runtime` jars. In addition, they may need to make sure these jars appear before other Hadoop jars in the order. Otherwise, classes may be loaded from the other non-shaded Hadoop jars and cause potential conflicts.
### How was this patch tested?
Relying on existing tests.
Closes #29843 from sunchao/SPARK-29250.
Authored-by: Chao Sun <sunchao@apple.com>
Signed-off-by: DB Tsai <d_tsai@apple.com>
2020-10-21 23:21:34 -04:00
|
|
|
jaxb-api/2.2.11//jaxb-api-2.2.11.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
jaxb-runtime/2.3.2//jaxb-runtime-2.3.2.jar
|
2020-05-04 11:14:12 -04:00
|
|
|
jcl-over-slf4j/1.7.30//jcl-over-slf4j-1.7.30.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
jdo-api/3.0.1//jdo-api-3.0.1.jar
|
2020-01-25 18:41:55 -05:00
|
|
|
jersey-client/2.30//jersey-client-2.30.jar
|
|
|
|
|
jersey-common/2.30//jersey-common-2.30.jar
|
|
|
|
|
jersey-container-servlet-core/2.30//jersey-container-servlet-core-2.30.jar
|
|
|
|
|
jersey-container-servlet/2.30//jersey-container-servlet-2.30.jar
|
|
|
|
|
jersey-hk2/2.30//jersey-hk2-2.30.jar
|
|
|
|
|
jersey-media-jaxb/2.30//jersey-media-jaxb-2.30.jar
|
|
|
|
|
jersey-server/2.30//jersey-server-2.30.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
jetty-sslengine/6.1.26//jetty-sslengine-6.1.26.jar
|
|
|
|
|
jetty-util/6.1.26//jetty-util-6.1.26.jar
|
|
|
|
|
jetty/6.1.26//jetty-6.1.26.jar
|
|
|
|
|
jline/2.14.6//jline-2.14.6.jar
|
|
|
|
|
joda-time/2.10.5//joda-time-2.10.5.jar
|
|
|
|
|
jodd-core/3.5.2//jodd-core-3.5.2.jar
|
|
|
|
|
jpam/1.1//jpam-1.1.jar
|
|
|
|
|
json/1.8//json-1.8.jar
|
2020-07-25 23:34:31 -04:00
|
|
|
json4s-ast_2.12/3.7.0-M5//json4s-ast_2.12-3.7.0-M5.jar
|
|
|
|
|
json4s-core_2.12/3.7.0-M5//json4s-core_2.12-3.7.0-M5.jar
|
|
|
|
|
json4s-jackson_2.12/3.7.0-M5//json4s-jackson_2.12-3.7.0-M5.jar
|
|
|
|
|
json4s-scalap_2.12/3.7.0-M5//json4s-scalap_2.12-3.7.0-M5.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
jsp-api/2.1//jsp-api-2.1.jar
|
|
|
|
|
jsr305/3.0.0//jsr305-3.0.0.jar
|
|
|
|
|
jta/1.1//jta-1.1.jar
|
2020-05-04 11:14:12 -04:00
|
|
|
jul-to-slf4j/1.7.30//jul-to-slf4j-1.7.30.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
kryo-shaded/4.0.2//kryo-shaded-4.0.2.jar
|
2020-11-03 01:23:26 -05:00
|
|
|
kubernetes-client/4.11.1//kubernetes-client-4.11.1.jar
|
|
|
|
|
kubernetes-model-admissionregistration/4.11.1//kubernetes-model-admissionregistration-4.11.1.jar
|
|
|
|
|
kubernetes-model-apiextensions/4.11.1//kubernetes-model-apiextensions-4.11.1.jar
|
|
|
|
|
kubernetes-model-apps/4.11.1//kubernetes-model-apps-4.11.1.jar
|
|
|
|
|
kubernetes-model-autoscaling/4.11.1//kubernetes-model-autoscaling-4.11.1.jar
|
|
|
|
|
kubernetes-model-batch/4.11.1//kubernetes-model-batch-4.11.1.jar
|
|
|
|
|
kubernetes-model-certificates/4.11.1//kubernetes-model-certificates-4.11.1.jar
|
|
|
|
|
kubernetes-model-common/4.11.1//kubernetes-model-common-4.11.1.jar
|
|
|
|
|
kubernetes-model-coordination/4.11.1//kubernetes-model-coordination-4.11.1.jar
|
|
|
|
|
kubernetes-model-core/4.11.1//kubernetes-model-core-4.11.1.jar
|
|
|
|
|
kubernetes-model-discovery/4.11.1//kubernetes-model-discovery-4.11.1.jar
|
|
|
|
|
kubernetes-model-events/4.11.1//kubernetes-model-events-4.11.1.jar
|
|
|
|
|
kubernetes-model-extensions/4.11.1//kubernetes-model-extensions-4.11.1.jar
|
|
|
|
|
kubernetes-model-metrics/4.11.1//kubernetes-model-metrics-4.11.1.jar
|
|
|
|
|
kubernetes-model-networking/4.11.1//kubernetes-model-networking-4.11.1.jar
|
|
|
|
|
kubernetes-model-policy/4.11.1//kubernetes-model-policy-4.11.1.jar
|
|
|
|
|
kubernetes-model-rbac/4.11.1//kubernetes-model-rbac-4.11.1.jar
|
|
|
|
|
kubernetes-model-scheduling/4.11.1//kubernetes-model-scheduling-4.11.1.jar
|
|
|
|
|
kubernetes-model-settings/4.11.1//kubernetes-model-settings-4.11.1.jar
|
|
|
|
|
kubernetes-model-storageclass/4.11.1//kubernetes-model-storageclass-4.11.1.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
leveldbjni-all/1.8//leveldbjni-all-1.8.jar
|
|
|
|
|
libfb303/0.9.3//libfb303-0.9.3.jar
|
|
|
|
|
libthrift/0.12.0//libthrift-0.12.0.jar
|
|
|
|
|
log4j/1.2.17//log4j-1.2.17.jar
|
2020-09-30 22:00:18 -04:00
|
|
|
logging-interceptor/3.12.12//logging-interceptor-3.12.12.jar
|
2020-01-19 22:05:30 -05:00
|
|
|
lz4-java/1.7.1//lz4-java-1.7.1.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
machinist_2.12/0.6.8//machinist_2.12-0.6.8.jar
|
|
|
|
|
macro-compat_2.12/1.1.1//macro-compat_2.12-1.1.1.jar
|
|
|
|
|
mesos/1.4.0/shaded-protobuf/mesos-1.4.0-shaded-protobuf.jar
|
|
|
|
|
metrics-core/4.1.1//metrics-core-4.1.1.jar
|
|
|
|
|
metrics-graphite/4.1.1//metrics-graphite-4.1.1.jar
|
|
|
|
|
metrics-jmx/4.1.1//metrics-jmx-4.1.1.jar
|
|
|
|
|
metrics-json/4.1.1//metrics-json-4.1.1.jar
|
|
|
|
|
metrics-jvm/4.1.1//metrics-jvm-4.1.1.jar
|
|
|
|
|
minlog/1.3.0//minlog-1.3.0.jar
|
2020-08-02 19:46:11 -04:00
|
|
|
netty-all/4.1.51.Final//netty-all-4.1.51.Final.jar
|
2020-07-23 19:20:17 -04:00
|
|
|
objenesis/2.6//objenesis-2.6.jar
|
2020-09-30 22:00:18 -04:00
|
|
|
okhttp/3.12.12//okhttp-3.12.12.jar
|
2020-07-23 19:20:17 -04:00
|
|
|
okio/1.14.0//okio-1.14.0.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
opencsv/2.3//opencsv-2.3.jar
|
2020-11-03 01:23:26 -05:00
|
|
|
openshift-model/4.11.1//openshift-model-4.11.1.jar
|
2020-10-02 03:06:03 -04:00
|
|
|
orc-core/1.5.12//orc-core-1.5.12.jar
|
|
|
|
|
orc-mapreduce/1.5.12//orc-mapreduce-1.5.12.jar
|
|
|
|
|
orc-shims/1.5.12//orc-shims-1.5.12.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
oro/2.0.8//oro-2.0.8.jar
|
|
|
|
|
osgi-resource-locator/1.0.3//osgi-resource-locator-1.0.3.jar
|
|
|
|
|
paranamer/2.8//paranamer-2.8.jar
|
|
|
|
|
parquet-column/1.10.1//parquet-column-1.10.1.jar
|
|
|
|
|
parquet-common/1.10.1//parquet-common-1.10.1.jar
|
|
|
|
|
parquet-encoding/1.10.1//parquet-encoding-1.10.1.jar
|
|
|
|
|
parquet-format/2.4.0//parquet-format-2.4.0.jar
|
|
|
|
|
parquet-hadoop/1.10.1//parquet-hadoop-1.10.1.jar
|
|
|
|
|
parquet-jackson/1.10.1//parquet-jackson-1.10.1.jar
|
|
|
|
|
protobuf-java/2.5.0//protobuf-java-2.5.0.jar
|
2020-02-20 12:08:33 -05:00
|
|
|
py4j/0.10.9//py4j-0.10.9.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
pyrolite/4.30//pyrolite-4.30.jar
|
|
|
|
|
scala-collection-compat_2.12/2.1.1//scala-collection-compat_2.12-2.1.1.jar
|
|
|
|
|
scala-compiler/2.12.10//scala-compiler-2.12.10.jar
|
|
|
|
|
scala-library/2.12.10//scala-library-2.12.10.jar
|
|
|
|
|
scala-parser-combinators_2.12/1.1.2//scala-parser-combinators_2.12-1.1.2.jar
|
|
|
|
|
scala-reflect/2.12.10//scala-reflect-2.12.10.jar
|
|
|
|
|
scala-xml_2.12/1.2.0//scala-xml_2.12-1.2.0.jar
|
|
|
|
|
shapeless_2.12/2.3.3//shapeless_2.12-2.3.3.jar
|
2020-07-25 13:58:25 -04:00
|
|
|
shims/0.9.0//shims-0.9.0.jar
|
2020-05-04 11:14:12 -04:00
|
|
|
slf4j-api/1.7.30//slf4j-api-1.7.30.jar
|
|
|
|
|
slf4j-log4j12/1.7.30//slf4j-log4j12-1.7.30.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
snakeyaml/1.24//snakeyaml-1.24.jar
|
2020-10-21 16:04:39 -04:00
|
|
|
snappy-java/1.1.8//snappy-java-1.1.8.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
spire-macros_2.12/0.17.0-M1//spire-macros_2.12-0.17.0-M1.jar
|
|
|
|
|
spire-platform_2.12/0.17.0-M1//spire-platform_2.12-0.17.0-M1.jar
|
|
|
|
|
spire-util_2.12/0.17.0-M1//spire-util_2.12-0.17.0-M1.jar
|
|
|
|
|
spire_2.12/0.17.0-M1//spire_2.12-0.17.0-M1.jar
|
|
|
|
|
stax-api/1.0.1//stax-api-1.0.1.jar
|
|
|
|
|
stream/2.9.6//stream-2.9.6.jar
|
|
|
|
|
super-csv/2.2.0//super-csv-2.2.0.jar
|
2020-01-31 20:41:27 -05:00
|
|
|
threeten-extra/1.5.0//threeten-extra-1.5.0.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
transaction-api/1.1//transaction-api-1.1.jar
|
2020-08-25 11:25:58 -04:00
|
|
|
univocity-parsers/2.9.0//univocity-parsers-2.9.0.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
velocity/1.5//velocity-1.5.jar
|
|
|
|
|
xbean-asm7-shaded/4.15//xbean-asm7-shaded-4.15.jar
|
2020-03-03 10:27:18 -05:00
|
|
|
xercesImpl/2.12.0//xercesImpl-2.12.0.jar
|
2020-03-05 19:39:02 -05:00
|
|
|
xml-apis/1.4.01//xml-apis-1.4.01.jar
|
[SPARK-30491][INFRA] Enable dependency audit files to tell dependency classifier
### What changes were proposed in this pull request?
Enable dependency audit files to tell the value of artifact id, version, and classifier of a dependency.
For example, `avro-mapred-1.8.2-hadoop2.jar` should be expanded to `avro-mapred/1.8.2/hadoop2/avro-mapred-1.8.2-hadoop2.jar` where `avro-mapred` is the artifact id, `1.8.2` is the version, and `haddop2` is the classifier.
### Why are the changes needed?
Dependency audit files are expected to be consumed by automated tests or downstream tools.
However, current dependency audit files under `dev/deps` only show jar names. And there isn't a simple rule on how to parse the jar name to get the values of different fields. For example, `hadoop2` is the classifier of `avro-mapred-1.8.2-hadoop2.jar`, in contrast, `incubating` is the version of `htrace-core-3.1.0-incubating.jar`.
Reference: There is a good example of the downstream tool that would be enabled as yhuai suggested,
> Say we have a Spark application that depends on a third-party dependency `foo`, which pulls in `jackson` as a transient dependency. Unfortunately, `foo` depends on a different version of `jackson` than Spark. So, in the pom of this Spark application, we use the dependency management section to pin the version of `jackson`. By doing this, we are lifting `jackson` to the top-level dependency of my application and I want to have a way to keep tracking what Spark uses. What we can do is to cross-check my Spark application's classpath with what Spark uses. Then, with a test written in my code base, whenever my application bumps Spark version, this test will check what we define in the application and what Spark has, and then remind us to change our application's pom if needed. In my case, I am fine to directly access git to get these audit files.
### Does this PR introduce any user-facing change?
No.
### How was this patch tested?
Code changes are verified by generated dependency audit files naturally. Thus, there are no tests added.
Closes #27177 from mengCareers/depsOptimize.
Lead-authored-by: Xinrong Meng <meng.careers@gmail.com>
Co-authored-by: mengCareers <meng.careers@gmail.com>
Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
2020-01-15 23:19:44 -05:00
|
|
|
xmlenc/0.52//xmlenc-0.52.jar
|
|
|
|
|
xz/1.5//xz-1.5.jar
|
|
|
|
|
zjsonpatch/0.3.0//zjsonpatch-0.3.0.jar
|
|
|
|
|
zookeeper/3.4.14//zookeeper-3.4.14.jar
|
2020-10-12 03:27:53 -04:00
|
|
|
zstd-jni/1.4.5-6//zstd-jni-1.4.5-6.jar
|
