Log RPC requests when they fails to improve error messages
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
For example, BigQuery sometime doesn't provide enough information in the respond. Users can only know what is invalid in the request, if we log the it.
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request
{
"code" : 400,
"errors" : [ {
"domain" : "global",
"message" : "Invalid value for: String is not a valid value",
"reason" : "invalid"
} ],
"message" : "Invalid value for: String is not a valid value"
}
at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:321)
at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1065)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:419)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:352)
at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:469)
at com.google.cloud.dataflow.sdk.util.BigQueryTableInserter.tryCreateTable(BigQueryTableInserter.java:427)
at com.google.cloud.dataflow.sdk.util.BigQueryTableInserter.tryCreateTable(BigQueryTableInserter.java:416)
at com.google.cloud.dataflow.sdk.util.BigQueryTableInserter.getOrCreateTable(BigQueryTableInserter.java:373)
at com.google.cloud.dataflow.sdk.io.BigQueryIO$StreamingWriteFn.getOrCreateTable(BigQueryIO.java:2127)
at com.google.cloud.dataflow.sdk.io.BigQueryIO$StreamingWriteFn.finishBundle(BigQueryIO.java:2099)
at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.finishBundle(DoFnRunnerBase.java:158)
Imported from Jira [BEAM-482](https://issues.apache.org/jira/browse/BEAM-482). Original Jira may contain additional context.
Reported by: peihe0@gmail.com.
Contributor guide
Research direction
Start by tracing the failed request path through BigQueryTableInserter and BigQueryIO, using the Google API exception in the report as the failure example. Determine where the RPC failure is handled and what request details are available there. Done means failed requests provide enough logged information to diagnose invalid values without relying only on the exception message.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- google-cloud, java
- Domain
- cloud, observability
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100