apache / apache/arrow-java

Arrow 13.0.0: org.apache.arrow.flight.FlightRuntimeException: CANCELLED: Failed to stream message

Open
#170 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Hello All,

I would like to ask your help in the following issue I am facing with.

I have a Java and a Python application in different pods running on Kubernetes. A running Arrow Flight server is on the Python side.

Arrow version: 13.0.0 (everything worked fine with version 12.0.1)

**flightClient.handshake();** throws this error and the method is in an infinite loop:

```BASH
org.apache.arrow.flight.FlightRuntimeException: CANCELLED: Failed to stream message
at org.apache.arrow.flight.CallStatus.toRuntimeException(CallStatus.java:131)
at org.apache.arrow.flight.grpc.StatusUtils.fromGrpcRuntimeException(StatusUtils.java:164)
at org.apache.arrow.flight.grpc.StatusUtils.fromThrowable(StatusUtils.java:185)
at org.apache.arrow.flight.auth2.ClientHandshakeWrapper.doClientHandshake(ClientHandshakeWrapper.java:59)
at org.apache.arrow.flight.FlightClient.handshake(FlightClient.java:212)
at ...
Caused by: io.grpc.StatusRuntimeException: INTERNAL: Failed to frame message
at io.grpc.Status.asRuntimeException(Status.java:530)
at io.grpc.internal.MessageFramer.writePayload(MessageFramer.java:154)
at io.grpc.internal.AbstractStream.writeMessage(AbstractStream.java:66)
at io.grpc.internal.ForwardingClientStream.writeMessage(ForwardingClientStream.java:37)
at io.grpc.internal.DelayedStream.writeMessage(DelayedStream.java:278)
at io.grpc.internal.RetriableStream$1SendMessageEntry.runWith(RetriableStream.java:567)
at io.grpc.internal.RetriableStream.delayOrExecute(RetriableStream.java:544)
at io.grpc.internal.RetriableStream.sendMessage(RetriableStream.java:575)
at io.grpc.internal.ClientCallImpl.sendMessageInternal(ClientCallImpl.java:525)
at io.grpc.internal.ClientCallImpl.sendMessage(ClientCallImpl.java:513)
at io.grpc.ForwardingClientCall.sendMessage(ForwardingClientCall.java:37)
at io.grpc.stub.ClientCalls$CallToStreamObserverAdapter.onNext(ClientCalls.java:378)
at org.apache.arrow.flight.auth2.ClientHandshakeWrapper.doClientHandshake(ClientHandshakeWrapper.java:48)
... 21 more
Caused by: java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses.
at com.google.protobuf.GeneratedMessageV3.getUnknownFields(GeneratedMessageV3.java:302)
at org.apache.arrow.flight.impl.Flight$HandshakeRequest.getSerializedSize(Flight.java:328)
at io.grpc.protobuf.lite.ProtoInputStream.available(ProtoInputStream.java:108)
at io.grpc.internal.MessageFramer.getKnownLength(MessageFramer.java:208)
at io.grpc.internal.MessageFramer.writePayload(MessageFramer.java:138)
... 32 more
```

### Component(s)

Java

Contributor guide

Open the contributing guide

Research direction

Start at org.apache.arrow.flight.auth2.ClientHandshakeWrapper.doClientHandshake and the generated Flight.HandshakeRequest serialization shown in the stack trace. Compare the Java client behavior and dependencies between Arrow 12.0.1 and 13.0.0, reproducing the handshake between the Java and Python applications in Kubernetes. Done means flightClient.handshake() completes without the framing error or infinite retry loop.

Written by the indexing model from the issue text.

Assessment

Tech stack
grpc, java, kubernetes, python
Domain
api, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.