[Bug]: WriteToBigquery Deadletter pattern does not work with FILE_LOADS method
Open
bug
gcp
io
new feature
P2
python
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What happened?
The Bigquery Deadletter pattern: https://beam.apache.org/documentation/patterns/bigqueryio/
Does not work on `WriteToBigquery` if `FILE_LOADS` strategy is used. The `insert_retry_strategy` parameter is not passed to [BigQueryBatchFileLoads](https://github.com/apache/beam/blob/b8ca0819529e0bafaae0c08abec7c4e5682d6b50/sdks/python/apache_beam/io/gcp/bigquery.py#L2363-L2380) and the output does not have a `'FailedRows'` tag.
The deadletter pattern is also important in batch systems - to prevent a single malformed row from failing the entire job.
### Issue Priority
Priority: 2
### Issue Component
Component: io-py-gcp
Contributor guide
Assessment
This issue has not been assessed yet.