apache / apache/pinot

Multistage: connect(..) failed: Address family not supported by protocol release v1.0.0

Open
#11,828 6 comments 0 reactions 0 assignees View on GitHub
stale troubleshooting
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
2d 3h
Merged PRs (30d)
195

Description

Pinot1.0.0 using multistage v2 option deployed from distribution, gets error running query.

Can see server/broker listening on the query and mailbox ports ok but client cannot connect to query.

Tried switching netty native v java implementation = no difference
Tried prefer ipv4 = no difference apart from the query showed that multistage was not enabled until the cluster config enable multistage was added in.

Online search suggests a shading issue with the grpc library in the distro.
https://stackoverflow.com/questions/73286776/grpc-unsupportedaddresstypeexception-but-only-when-packaged-with-shadowjar

```
QueryExecutionError:
Error dispatching query to server=yourserver.com@{10124,10126} stage=2
org.apache.pinot.query.service.dispatch.QueryDispatcher.submit(QueryDispatcher.java:144)
org.apache.pinot.query.service.dispatch.QueryDispatcher.submitAndReduce(QueryDispatcher.java:93)
org.apache.pinot.broker.requesthandler.MultiStageBrokerRequestHandler.handleRequest(MultiStageBrokerRequestHandler.java:179)
org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:263)
UNAVAILABLE: io exception
io.grpc.Status.asRuntimeException(Status.java:539)
io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:487)
io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:576)
io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
connect(..) failed: Address family not supported by protocol: /yourserver.com:10124
connect(..) failed: Address family not supported by protocol
io.grpc.netty.shaded.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155)
io.grpc.netty.shaded.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128)
io.grpc.netty.shaded.io.netty.channel.unix.Socket.connect(Socket.java:313)
io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel.doConnect0(AbstractEpollChannel.java:773)
connect(..) failed: Address family not supported by protocol
io.grpc.netty.shaded.io.netty.channel.unix.Errors.newConnectException0(Errors.java:155)
io.grpc.netty.shaded.io.netty.channel.unix.Errors.handleConnectErrno(Errors.java:128)
io.grpc.netty.shaded.io.netty.channel.unix.Socket.connect(Socket.java:313)
io.grpc.netty.shaded.io.netty.channel.epoll.AbstractEpollChannel.doConnect0(AbstractEpollChannel.java:773)
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the Pinot 1.0.0 distribution, multistage v2, and the reported query, then start from QueryDispatcher.java:144 and the MultiStageBrokerRequestHandler.java:179 stack-trace entries. Inspect the packaged gRPC/Netty dependencies and compare them with the non-shaded implementation; done means multistage queries can connect to the query and mailbox ports without the address-family error.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, java
Domain
distributed-systems, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.