The dataflow failure log doesn't include error message
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
In a customer case investigation, we got this from the log:
Append to stream projects/optimizely-edp-prod-data-1001/datasets/xxxx/tables/conversions_attributes/streams/_default by client #0 failed with error, operations will be retried. Details: com.google.cloud.bigquery.storage.v1.ConnectionWorker.requestCallback(ConnectionWorker.java:656) com.google.cloud.bigquery.storage.v1.ConnectionWorker.access$100(ConnectionWorker.java:58) com.google.cloud.bigquery.storage.v1.ConnectionWorker$2.run(ConnectionWorker.java:246) com.google.cloud.bigquery.storage.v1.StreamConnection$1.onResponse(StreamConnection.java:63) com.google.cloud.bigquery.storage.v1.StreamConnection$1.onResponse(StreamConnection.java:54) com.google.api.gax.tracing.TracedResponseObserver.onResponse(TracedResponseObserver.java:90) com.google.api.gax.grpc.ExceptionResponseObserver.onResponseImpl(ExceptionResponseObserver.java:74) com.google.api.gax.rpc.StateCheckingResponseObserver.onResponse(StateCheckingResponseObserver.java:60) com.google.api.gax.grpc.GrpcDirectStreamController$ResponseObserverAdapter.onMessage(GrpcDirectStreamController.java:134) io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33) io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33) io.grpc.ForwardingClientCallListener.onMessage(ForwardingClientCallListener.java:33) io.grpc.internal.DelayedClientCall$DelayedListener.onMessage(DelayedClientCall.java:454) io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInternal(ClientCallImpl.java:662) io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:647) io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:748),no error
It seems only the stacktrace was provided but not the error message. It makes understanding what's going on very difficult. Could we fix the error logging to include error message?
Contributor guide
Research direction
The report names ConnectionWorker.requestCallback and related Java stack frames, but no Beam source file or test. Start by tracing the Dataflow/BigQuery Storage write failure logging around that callback; done means the emitted failure log contains the error message as well as the stack trace.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100