apache / apache/maven-dependency-plugin

[MDEP-640] Could not get correct module classpath with -pl option

Open
#1,072 1 comment 0 reactions 0 assignees View on GitHub
bug priority:major
Dominant language
Java
Stars
175
Forks
196
Avg merge
19h 30m
Merged PRs (30d)
5

Description

**[Yuming Wang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=q79969786)** opened **[MDEP-640](https://issues.apache.org/jira/browse/MDEP-640?redirect=false)** and commented

`build-classpath` could not get correct classpath with `-pl`:

```java
$ git clone https://github.com/wangyum/spark.git
$ cd spark && git checkout CARMEL-363
$ build/mvn -q versions:set -DnewVersion=CARMEL-363 -DgenerateBackupPoms=false  -Phadoop-3.1
$ build/mvn -Phadoop-3.1 jar:jar jar:test-jar install:install clean
$ build/mvn -Phadoop-3.1 dependency:build-classpath -pl assembly
```

It always add `orc-core-1.5.4-nohive.jar` and `orc-mapreduce-1.5.4-nohive.jar` to classpath:

```
RoaringBitmap-0.5.11.jar
accessors-smart-1.2.jar
activation-1.1.1.jar
aircompressor-0.10.jar
antlr4-runtime-4.7.1.jar
aopalliance-repackaged-2.4.0-b34.jar
arpack_combined_all-0.1.jar
arrow-format-0.12.0.jar
arrow-memory-0.12.0.jar
arrow-vector-0.12.0.jar
audience-annotations-0.5.0.jar
avro-1.8.2.jar
avro-ipc-1.8.2.jar
avro-mapred-1.8.2-hadoop2.jar
breeze-macros_2.12-0.13.2.jar
breeze_2.12-0.13.2.jar
chill-java-0.9.3.jar
chill_2.12-0.9.3.jar
commons-beanutils-1.9.3.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-3.0.11.jar
commons-compress-1.9.jar
commons-configuration2-2.1.1.jar
commons-crypto-1.0.0.jar
commons-io-2.4.jar
commons-lang-2.6.jar
commons-lang3-3.8.1.jar
commons-math3-3.4.1.jar
commons-net-3.1.jar
commons-text-1.6.jar
compress-lzf-1.0.3.jar
core-1.1.2.jar
curator-client-2.13.0.jar
curator-framework-2.13.0.jar
curator-recipes-2.13.0.jar
dnsjava-2.1.7.jar
flatbuffers-java-1.9.0.jar
gson-2.2.4.jar
guava-14.0.1.jar
hadoop-annotations-3.2.0.jar
hadoop-auth-3.2.0.jar
hadoop-client-3.2.0.jar
hadoop-common-3.2.0.jar
hadoop-hdfs-client-3.2.0.jar
hadoop-mapreduce-client-common-3.2.0.jar
hadoop-mapreduce-client-core-3.2.0.jar
hadoop-mapreduce-client-jobclient-3.2.0.jar
hadoop-yarn-api-3.2.0.jar
hadoop-yarn-client-3.2.0.jar
hadoop-yarn-common-3.2.0.jar
hive-common-2.3.4.jar
hive-llap-client-2.3.4.jar
hive-llap-common-2.3.4.jar
hive-serde-2.3.4.jar
hive-service-rpc-2.3.4.jar
hive-shims-0.23-2.3.4.jar
hive-shims-2.3.4.jar
hive-shims-common-2.3.4.jar
hive-shims-scheduler-2.3.4.jar
hive-storage-api-2.6.0.jar
hk2-api-2.4.0-b34.jar
hk2-locator-2.4.0-b34.jar
hk2-utils-2.4.0-b34.jar
hppc-0.7.2.jar
htrace-core4-4.1.0-incubating.jar
httpclient-4.5.6.jar
httpcore-4.4.10.jar
ivy-2.4.0.jar
jackson-annotations-2.9.6.jar
jackson-core-2.9.6.jar
jackson-core-asl-1.9.13.jar
jackson-databind-2.9.6.jar
jackson-jaxrs-base-2.9.5.jar
jackson-jaxrs-json-provider-2.9.5.jar
jackson-mapper-asl-1.9.13.jar
jackson-module-jaxb-annotations-2.9.6.jar
jackson-module-paranamer-2.9.6.jar
jackson-module-scala_2.12-2.9.6.jar
janino-3.0.11.jar
javassist-3.18.1-GA.jar
javax.annotation-api-1.2.jar
javax.inject-2.4.0-b34.jar
javax.servlet-api-3.1.0.jar
javax.ws.rs-api-2.0.1.jar
jaxb-api-2.2.11.jar
jcip-annotations-1.0-1.jar
jcl-over-slf4j-1.7.16.jar
jersey-client-2.22.2.jar
jersey-common-2.22.2.jar
jersey-container-servlet-2.22.2.jar
jersey-container-servlet-core-2.22.2.jar
jersey-guava-2.22.2.jar
jersey-media-jaxb-2.22.2.jar
jersey-server-2.22.2.jar
jetty-webapp-9.3.24.v20180605.jar
jetty-xml-9.3.24.v20180605.jar
jline-2.14.6.jar
joda-time-2.9.3.jar
json-1.8.jar
json-smart-2.3.jar
json4s-ast_2.12-3.5.3.jar
json4s-core_2.12-3.5.3.jar
json4s-jackson_2.12-3.5.3.jar
json4s-scalap_2.12-3.5.3.jar
jsp-api-2.1.jar
jsr305-3.0.0.jar
jtransforms-2.4.0.jar
jul-to-slf4j-1.7.16.jar
kerb-admin-1.0.1.jar
kerb-client-1.0.1.jar
kerb-common-1.0.1.jar
kerb-core-1.0.1.jar
kerb-crypto-1.0.1.jar
kerb-identity-1.0.1.jar
kerb-server-1.0.1.jar
kerb-simplekdc-1.0.1.jar
kerb-util-1.0.1.jar
kerby-asn1-1.0.1.jar
kerby-config-1.0.1.jar
kerby-pkix-1.0.1.jar
kerby-util-1.0.1.jar
kerby-xdr-1.0.1.jar
kryo-shaded-4.0.2.jar
leveldbjni-all-1.8.jar
log4j-1.2.17.jar
lz4-java-1.5.0.jar
machinist_2.12-0.6.1.jar
macro-compat_2.12-1.1.1.jar
metrics-core-3.1.5.jar
metrics-graphite-3.1.5.jar
metrics-json-3.1.5.jar
metrics-jvm-3.1.5.jar
minlog-1.3.0.jar
netty-3.9.9.Final.jar
netty-all-4.1.30.Final.jar
nimbus-jose-jwt-4.41.1.jar
objenesis-2.5.1.jar
okhttp-2.7.5.jar
okio-1.6.0.jar
opencsv-2.3.jar
orc-core-1.5.4-nohive.jar
orc-core-1.5.4.jar
orc-mapreduce-1.5.4-nohive.jar
orc-shims-1.5.4.jar
oro-2.0.8.jar
osgi-resource-locator-1.0.1.jar
paranamer-2.8.jar
parquet-column-1.10.1.jar
parquet-common-1.10.1.jar
parquet-encoding-1.10.1.jar
parquet-format-2.4.0.jar
parquet-hadoop-1.10.1.jar
parquet-jackson-1.10.1.jar
protobuf-java-2.5.0.jar
py4j-0.10.8.1.jar
pyrolite-4.13.jar
re2j-1.1.jar
scala-compiler-2.12.8.jar
scala-library-2.12.8.jar
scala-parser-combinators_2.12-1.1.0.jar
scala-reflect-2.12.8.jar
scala-xml_2.12-1.0.5.jar
shapeless_2.12-2.3.2.jar
slf4j-api-1.7.16.jar
slf4j-log4j12-1.7.16.jar
snappy-java-1.1.7.1.jar
spark-catalyst_2.12-CARMEL-363.jar
spark-core_2.12-CARMEL-363.jar
spark-graphx_2.12-CARMEL-363.jar
spark-kvstore_2.12-CARMEL-363.jar
spark-launcher_2.12-CARMEL-363.jar
spark-mllib-local_2.12-CARMEL-363.jar
spark-mllib_2.12-CARMEL-363.jar
spark-network-common_2.12-CARMEL-363.jar
spark-network-shuffle_2.12-CARMEL-363.jar
spark-repl_2.12-CARMEL-363.jar
spark-sketch_2.12-CARMEL-363.jar
spark-sql_2.12-CARMEL-363.jar
spark-streaming_2.12-CARMEL-363.jar
spark-tags_2.12-CARMEL-363.jar
spark-unsafe_2.12-CARMEL-363.jar
spire-macros_2.12-0.13.0.jar
spire_2.12-0.13.0.jar
stax2-api-3.1.4.jar
stream-2.7.0.jar
token-provider-1.0.1.jar
univocity-parsers-2.7.3.jar
validation-api-1.1.0.Final.jar
woodstox-core-5.0.3.jar
xbean-asm7-shaded-4.12.jar
xz-1.5.jar
zookeeper-3.4.13.jar
zstd-jni-1.3.2-2.jar
```

But if remove `-pl assembly` and navigator to _org.apache.spark:spark-assembly_2.12_.

```sh
build/mvn -Phadoop-3.1 dependency:build-classpath
```

The classpath is:

```
RoaringBitmap-0.5.11.jar
accessors-smart-1.2.jar
activation-1.1.1.jar
aircompressor-0.10.jar
antlr4-runtime-4.7.1.jar
aopalliance-repackaged-2.4.0-b34.jar
arpack_combined_all-0.1.jar
arrow-format-0.12.0.jar
arrow-memory-0.12.0.jar
arrow-vector-0.12.0.jar
audience-annotations-0.5.0.jar
avro-1.8.2.jar
avro-ipc-1.8.2.jar
avro-mapred-1.8.2-hadoop2.jar
breeze-macros_2.12-0.13.2.jar
breeze_2.12-0.13.2.jar
chill-java-0.9.3.jar
chill_2.12-0.9.3.jar
commons-beanutils-1.9.3.jar
commons-cli-1.2.jar
commons-codec-1.10.jar
commons-collections-3.2.2.jar
commons-compiler-3.0.11.jar
commons-compress-1.9.jar
commons-configuration2-2.1.1.jar
commons-crypto-1.0.0.jar
commons-io-2.4.jar
commons-lang-2.6.jar
commons-lang3-3.8.1.jar
commons-math3-3.4.1.jar
commons-net-3.1.jar
commons-text-1.6.jar
compress-lzf-1.0.3.jar
core-1.1.2.jar
curator-client-2.13.0.jar
curator-framework-2.13.0.jar
curator-recipes-2.13.0.jar
dnsjava-2.1.7.jar
flatbuffers-java-1.9.0.jar
gson-2.2.4.jar
guava-14.0.1.jar
hadoop-annotations-3.2.0.jar
hadoop-auth-3.2.0.jar
hadoop-client-3.2.0.jar
hadoop-common-3.2.0.jar
hadoop-hdfs-client-3.2.0.jar
hadoop-mapreduce-client-common-3.2.0.jar
hadoop-mapreduce-client-core-3.2.0.jar
hadoop-mapreduce-client-jobclient-3.2.0.jar
hadoop-yarn-api-3.2.0.jar
hadoop-yarn-client-3.2.0.jar
hadoop-yarn-common-3.2.0.jar
hive-common-2.3.4.jar
hive-llap-client-2.3.4.jar
hive-llap-common-2.3.4.jar
hive-serde-2.3.4.jar
hive-service-rpc-2.3.4.jar
hive-shims-0.23-2.3.4.jar
hive-shims-2.3.4.jar
hive-shims-common-2.3.4.jar
hive-shims-scheduler-2.3.4.jar
hive-storage-api-2.6.0.jar
hk2-api-2.4.0-b34.jar
hk2-locator-2.4.0-b34.jar
hk2-utils-2.4.0-b34.jar
hppc-0.7.2.jar
htrace-core4-4.1.0-incubating.jar
httpclient-4.5.6.jar
httpcore-4.4.10.jar
ivy-2.4.0.jar
jackson-annotations-2.9.6.jar
jackson-core-2.9.6.jar
jackson-core-asl-1.9.13.jar
jackson-databind-2.9.6.jar
jackson-jaxrs-base-2.9.5.jar
jackson-jaxrs-json-provider-2.9.5.jar
jackson-mapper-asl-1.9.13.jar
jackson-module-jaxb-annotations-2.9.6.jar
jackson-module-paranamer-2.9.6.jar
jackson-module-scala_2.12-2.9.6.jar
janino-3.0.11.jar
javassist-3.18.1-GA.jar
javax.annotation-api-1.2.jar
javax.inject-2.4.0-b34.jar
javax.servlet-api-3.1.0.jar
javax.ws.rs-api-2.0.1.jar
jaxb-api-2.2.11.jar
jcip-annotations-1.0-1.jar
jcl-over-slf4j-1.7.16.jar
jersey-client-2.22.2.jar
jersey-common-2.22.2.jar
jersey-container-servlet-2.22.2.jar
jersey-container-servlet-core-2.22.2.jar
jersey-guava-2.22.2.jar
jersey-media-jaxb-2.22.2.jar
jersey-server-2.22.2.jar
jetty-webapp-9.3.24.v20180605.jar
jetty-xml-9.3.24.v20180605.jar
jline-2.14.6.jar
joda-time-2.9.3.jar
json-1.8.jar
json-smart-2.3.jar
json4s-ast_2.12-3.5.3.jar
json4s-core_2.12-3.5.3.jar
json4s-jackson_2.12-3.5.3.jar
json4s-scalap_2.12-3.5.3.jar
jsp-api-2.1.jar
jsr305-3.0.0.jar
jtransforms-2.4.0.jar
jul-to-slf4j-1.7.16.jar
kerb-admin-1.0.1.jar
kerb-client-1.0.1.jar
kerb-common-1.0.1.jar
kerb-core-1.0.1.jar
kerb-crypto-1.0.1.jar
kerb-identity-1.0.1.jar
kerb-server-1.0.1.jar
kerb-simplekdc-1.0.1.jar
kerb-util-1.0.1.jar
kerby-asn1-1.0.1.jar
kerby-config-1.0.1.jar
kerby-pkix-1.0.1.jar
kerby-util-1.0.1.jar
kerby-xdr-1.0.1.jar
kryo-shaded-4.0.2.jar
leveldbjni-all-1.8.jar
log4j-1.2.17.jar
lz4-java-1.5.0.jar
machinist_2.12-0.6.1.jar
macro-compat_2.12-1.1.1.jar
metrics-core-3.1.5.jar
metrics-graphite-3.1.5.jar
metrics-json-3.1.5.jar
metrics-jvm-3.1.5.jar
minlog-1.3.0.jar
netty-3.9.9.Final.jar
netty-all-4.1.30.Final.jar
nimbus-jose-jwt-4.41.1.jar
objenesis-2.5.1.jar
okhttp-2.7.5.jar
okio-1.6.0.jar
opencsv-2.3.jar
orc-core-1.5.4.jar
orc-mapreduce-1.5.4.jar
orc-shims-1.5.4.jar
oro-2.0.8.jar
osgi-resource-locator-1.0.1.jar
paranamer-2.8.jar
parquet-column-1.10.1.jar
parquet-common-1.10.1.jar
parquet-encoding-1.10.1.jar
parquet-format-2.4.0.jar
parquet-hadoop-1.10.1.jar
parquet-jackson-1.10.1.jar
protobuf-java-2.5.0.jar
py4j-0.10.8.1.jar
pyrolite-4.13.jar
re2j-1.1.jar
scala-compiler-2.12.8.jar
scala-library-2.12.8.jar
scala-parser-combinators_2.12-1.1.0.jar
scala-reflect-2.12.8.jar
scala-xml_2.12-1.0.5.jar
shapeless_2.12-2.3.2.jar
slf4j-api-1.7.16.jar
slf4j-log4j12-1.7.16.jar
snappy-java-1.1.7.1.jar
spark-catalyst_2.12-CARMEL-363.jar
spark-core_2.12-CARMEL-363.jar
spark-graphx_2.12-CARMEL-363.jar
spark-kvstore_2.12-CARMEL-363.jar
spark-launcher_2.12-CARMEL-363.jar
spark-mllib-local_2.12-CARMEL-363.jar
spark-mllib_2.12-CARMEL-363.jar
spark-network-common_2.12-CARMEL-363.jar
spark-network-shuffle_2.12-CARMEL-363.jar
spark-repl_2.12-CARMEL-363.jar
spark-sketch_2.12-CARMEL-363.jar
spark-sql_2.12-CARMEL-363.jar
spark-streaming_2.12-CARMEL-363.jar
spark-tags_2.12-CARMEL-363.jar
spark-unsafe_2.12-CARMEL-363.jar
spire-macros_2.12-0.13.0.jar
spire_2.12-0.13.0.jar
stax2-api-3.1.4.jar
stream-2.7.0.jar
token-provider-1.0.1.jar
univocity-parsers-2.7.3.jar
validation-api-1.1.0.Final.jar
woodstox-core-5.0.3.jar
xbean-asm7-shaded-4.12.jar
xz-1.5.jar
zookeeper-3.4.13.jar
zstd-jni-1.3.2-2.jar
```

---

**Affects:** 3.1.1

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the report with the shown Spark checkout and the two `build/mvn dependency:build-classpath` commands, comparing `-pl assembly` with the `org.apache.spark:spark-assembly_2.12` invocation. Trace the `dependency:build-classpath` entry point and determine why the module selection includes the `-nohive` ORC artifacts; done means both commands produce the expected dependency set.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.