googleapis / googleapis/google-cloud-python
Refactor BulkWriter retries
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 1.8k
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 122
Description
There are a number of issues with the current BulkWriter retry implementation:
- the limit of 15 retries is [hard-coded](https://github.com/googleapis/python-firestore/blob/8ddb859c8178043c0dcd09c85d68ba5d311563bf/google/cloud/firestore_v1/bulk_writer.py#L341), not user configurable
- the BulkWriter-level retries doesn't limit the underlying gapi-clevel retries, resulting in potentially very slow retries
- no max backoff cap
- does not check for idempotency, meaning some operations may be retried when they shouldn't
- uses custom hand-rolled retry logic, instead of relying on the standard api_core
I sugest a full internal refactor of the class, keeping the API the same but cleaning up the internals, and surfacing some extra user configuration
Contributor guide
Assessment
This issue has not been assessed yet.