[dependency] The module `pulsar-zookeeper-utils` has potential conflict dependency problem
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
**Describe the bug**
The Pulsar broker has a potential conflict dependency problem.
When building the KoP protocol handler, we met this conflict dependency problem. See error logs below:
```
[INFO] StreamNative :: Pulsar Protocol Handler :: Kafka Client Factory SUCCESS [ 0.196 s]
[INFO] StreamNative :: Pulsar Protocol Handler :: Kafka ... FAILURE [ 14.864 s]
[INFO] StreamNative :: Pulsar Protocol Handler :: OAuth 2.0 Client SKIPPED
[INFO] Kafka Payload Processor (Original) ................. SKIPPED
[INFO] StreamNative :: Pulsar Protocol Handler :: KoP Tests SKIPPED
[INFO] Kafka Payload Processor ............................ SKIPPED
[INFO] Tests for shaded kafka-payload-processor module .... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 39.702 s
[INFO] Finished at: 2022-04-23T22:05:31Z
[INFO] ------------------------------------------------------------------------
Error: Failed to execute goal on project pulsar-protocol-handler-kafka: Could not resolve dependencies for project io.streamnative.pulsar.handlers:pulsar-protocol-handler-kafka:jar:2.9.2.14: Failed to collect dependencies for io.streamnative.pulsar.handlers:pulsar-protocol-handler-kafka:jar:2.9.2.14: Could not resolve version conflict among [io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> org.apache.bookkeeper:stream-storage-server:jar:4.14.4 -> org.apache.bookkeeper:stream-storage-java-client:jar:4.14.4 -> io.grpc:grpc-core:jar:1.42.1, io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> org.apache.bookkeeper:stream-storage-server:jar:4.14.4 -> org.apache.bookkeeper:stream-storage-java-client:jar:4.14.4 -> io.grpc:grpc-testing:jar:1.42.1 -> io.grpc:grpc-core:jar:[1.42.1,1.42.1], io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1], io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-grpclb:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1], io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-netty:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1], io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-rls:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1], io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-services:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1], io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 -> io.grpc:grpc-core:jar:1.45.1, io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 -> io.grpc:grpc-netty-shaded:jar:[1.45.1,1.45.1] -> io.grpc:grpc-core:jar:[1.45.1,1.45.1]] -> [Help 1]
```
After format, we could find that the module `pulsar-zookeeper-utils` may depend on different versions package `grpc-core`.
```
Cannot resolve Could not resolve version conflict among
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> org.apache.bookkeeper:stream-storage-server:jar:4.14.4 -> org.apache.bookkeeper:stream-storage-java-client:jar:4.14.4 -> io.grpc:grpc-core:jar:1.42.1
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> org.apache.bookkeeper:stream-storage-server:jar:4.14.4 -> org.apache.bookkeeper:stream-storage-java-client:jar:4.14.4 -> io.grpc:grpc-testing:jar:1.42.1 -> io.grpc:grpc-core:jar:[1.42.1,1.42.1]
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1]
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-grpclb:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1]
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-netty:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1]
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-rls:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1]
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-services:jar:1.45.1 -> io.grpc:grpc-core:jar:[1.45.1,1.45.1]
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 -> io.grpc:grpc-core:jar:1.45.1
io.streamnative:pulsar-broker:jar:2.9.2.14 -> io.streamnative:pulsar-zookeeper-utils:jar:2.9.2.14 -> io.grpc:grpc-all:jar:1.45.1 -> io.grpc:grpc-xds:jar:1.45.1 -> io.grpc:grpc-netty-shaded:jar:[1.45.1,1.45.1] -> io.grpc:grpc-core:jar:[1.45.1,1.45.1]
```
I think we should resolve the dependency of the module `pulsar-zookeeper-utils`.
**To Reproduce**
Steps to reproduce the behavior:
1. Create a new project
2. Depend on the package `io.streamnative:pulsar-broker:2.9.2.14`.
3. Try to build the project.
4. See error.
**Expected behavior**
The project should be built successfully.
**Context**
I think we could exclude all grpc-related packages from `org.apache.bookkeeper:stream-storage-java-client` in module `pulsar-zookeeper-utils` because it also import package `grpc-all`.
Contributor guide
Research direction
Start in the pulsar-zookeeper-utils module's dependency declarations and trace its org.apache.bookkeeper:stream-storage-java-client and grpc-all dependencies. Reproduce the failure by depending on io.streamnative:pulsar-broker:2.9.2.14 and building a new project. Done means the conflicting grpc-core versions no longer prevent the project from building.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- grpc, java
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100