apache / apache/uniffle

[Bug] Duplicated shaded-grpc jar cause task fail

Open
#1,548 3 comments 0 reactions 1 assignee Claimed by @lifeSo View on GitHub
Dominant language
Java
Stars
454
Forks
172
Avg merge
5d 17h
Merged PRs (30d)
5

Description

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

### Search before asking

- [X] I have searched in the [issues](https://github.com/apache/incubator-uniffle/issues?q=is%3Aissue) and found no similar issues.

### Describe the bug

There is shaded-grpc jar in client-tez,
When there is also shaded-grpc jar in user UDF,
then, there will be sometimes task fail.

![Uploading image.png…]()

### Affects Version(s)

0.7.0

### Uniffle Server Log Output

_No response_

### Uniffle Engine Log Output

```logtalk
###|||2024-02-23 11:07:41.996|||INFO|||-|||-|||main|||-|||RssDAGAppMaster--->Registering coordinators 172.19.193.73:19999,172.19.193.182:19999
###|||2024-02-23 11:07:41.996|||INFO|||-|||-|||main|||-|||CoordinatorClientFactory--->Start to create coordinator clients from 172.19.193.73:19999,172.19.193.182:19999
###|||2024-02-23 11:07:42.013|||ERROR|||-|||-|||main|||-|||RssDAGAppMaster--->Error starting RssDAGAppMaster
java.lang.NoSuchFieldError: EPOLL_DOMAIN_CLIENT_CHANNEL_TYPE
at io.grpc.netty.shaded.io.grpc.netty.UdsNettyChannelProvider.isAvailable(UdsNettyChannelProvider.java:34)
at io.grpc.ManagedChannelRegistry$ManagedChannelPriorityAccessor.isAvailable(ManagedChannelRegistry.java:172)
at io.grpc.ManagedChannelRegistry$ManagedChannelPriorityAccessor.isAvailable(ManagedChannelRegistry.java:168)
at io.grpc.ServiceProviders.loadAll(ServiceProviders.java:68)
at io.grpc.ManagedChannelRegistry.getDefaultRegistry(ManagedChannelRegistry.java:96)
at io.grpc.ManagedChannelProvider.provider(ManagedChannelProvider.java:41)
at io.grpc.ManagedChannelBuilder.forAddress(ManagedChannelBuilder.java:39)
at org.apache.uniffle.client.impl.grpc.GrpcClient.(GrpcClient.java:43)
at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.(CoordinatorGrpcClient.java:91)
at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.(CoordinatorGrpcClient.java:87)
at org.apache.uniffle.client.impl.grpc.CoordinatorGrpcClient.(CoordinatorGrpcClient.java:83)
at org.apache.uniffle.client.factory.CoordinatorClientFactory.createCoordinatorClient(CoordinatorClientFactory.java:43)
at org.apache.uniffle.client.factory.CoordinatorClientFactory.createCoordinatorClient(CoordinatorClientFactory.java:69)
at org.apache.uniffle.client.impl.ShuffleWriteClientImpl.registerCoordinators(ShuffleWriteClientImpl.java:431)
at org.apache.tez.dag.app.RssDAGAppMaster.initAndStartRSSClient(RssDAGAppMaster.java:144)
at org.apache.tez.dag.app.RssDAGAppMaster.serviceInit(RssDAGAppMaster.java:107)
at org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
at org.apache.tez.dag.app.RssDAGAppMaster.main(RssDAGAppMaster.java:348)
```

### Uniffle Server Configurations

_No response_

### Uniffle Engine Configurations

_No response_

### Additional context

_No response_

### Are you willing to submit PR?

- [X] Yes I am willing to submit a PR!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.