BigQueryIO Storage Write API method - suppress logging API errors for known use cases.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Many BigQuery Storage API errors get logged at ERROR level even though they are expected and handled by the Beam's transform. They should be suppressed, or output at DEBUG/INFO level to differentiate from abnormal behavior.
Example:
```
{
"jsonPayload":
{ "message": "Got error io.grpc.StatusRuntimeException: ALREADY_EXISTS: The offset
is within stream, expected offset 52125, received 51264 Entity: projects/event-processing-demo/datasets/bigquery_io/tables/events/streams/CiQ2MmZlOTFjNS0wMDAwLTIzNTItOWMxYS01ODI0MjlhOWRiOGM
closing projects/event-processing-demo/datasets/bigquery_io/tables/events/streams/CiQ2MmZlOTFjNS0wMDAwLTIzNTItOWMxYS01ODI0MjlhOWRiOGM",
"step": "Save Rows to BigQuery/StorageApiLoads/StorageApiWriteSharded/Write Records", "worker": "data-processing-streaming-10250846-kps9-harness-hnj3",
"stage": "P6", "job": "2021-10-25_08_46_20-4863073108555028756", "thread": "258", "work": "3800026c09938fac-5cf2f68daacdd",
"logger": "org.apache.beam.sdk.io.gcp.bigquery.StorageApiWritesShardedRecords" }
,
"resource":
{
"type": "dataflow_step",
"labels":
{ "project_id": "event-processing-demo", "step_id": "Save
Rows to BigQuery/StorageApiLoads/StorageApiWriteSharded/Write Records", "job_id": "2021-10-25_08_46_20-4863073108555028756",
"region": "us-central1", "job_name": "data-processing-streaming-storage-write-api-200-2" }
},
"timestamp":
"2021-10-25T15:57:43.617Z",
"severity": "ERROR",
"labels":
{ "compute.googleapis.com/resource_type":
"instance", "dataflow.googleapis.com/job_name": "data-processing-streaming-storage-write-api-200-2",
"dataflow.googleapis.com/log_type": "supportability", "compute.googleapis.com/resource_id": "6365565973148954518",
"dataflow.googleapis.com/job_id": "2021-10-25_08_46_20-4863073108555028756", "dataflow.googleapis.com/region":
"us-central1", "compute.googleapis.com/resource_name": "data-processing-streaming-10250846-kps9-harness-hnj3"
}
,
"logName": "projects/event-processing-demo/logs/dataflow.googleapis.com%2Fworker",
"receiveTimestamp":
"2021-10-25T15:57:47.557344849Z"
}
```
Additional exceptions that fit this category:
* Got error `io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is closed due to com.google.api.gax.rpc.UnavailableException: io.grpc.StatusRuntimeException: UNAVAILABLE: Connection closed after GOAWAY. HTTP/2 error code: NO_ERROR, debug data: server_shutting_down closing`
* Got error `com.google.api.gax.rpc.AbortedException: io.grpc.StatusRuntimeException: ABORTED: Closing the stream because server is restarted`. This is expected and client is advised to reconnect.
* Got error `io.grpc.StatusRuntimeException: FAILED_PRECONDITION: Stream is closed due to com.google.api.gax.rpc.AbortedException: io.grpc.StatusRuntimeException: ABORTED: Closing the stream because server is restarted`. This is expected and client is advised to reconnect.
Imported from Jira [BEAM-13162](https://issues.apache.org/jira/browse/BEAM-13162). Original Jira may contain additional context.
Reported by: slilichenko.
Contributor guide
Assessment
This issue has not been assessed yet.