googleapis / googleapis/google-cloud-python

Refactor BulkWriter retries

Open
#15,385 1 comment 0 reactions 1 assignee Claimed by @daniel-sanche View on GitHub
api: firestore priority: p3 type: cleanup
Dominant language
Python
Stars
5.4k
Forks
1.8k
Avg merge
3d 4h
Merged PRs (30d)
122

Description

There are a number of issues with the current BulkWriter retry implementation:
- the limit of 15 retries is [hard-coded](https://github.com/googleapis/python-firestore/blob/8ddb859c8178043c0dcd09c85d68ba5d311563bf/google/cloud/firestore_v1/bulk_writer.py#L341), not user configurable
- the BulkWriter-level retries doesn't limit the underlying gapi-clevel retries, resulting in potentially very slow retries
- no max backoff cap
- does not check for idempotency, meaning some operations may be retried when they shouldn't
- uses custom hand-rolled retry logic, instead of relying on the standard api_core

I sugest a full internal refactor of the class, keeping the API the same but cleaning up the internals, and surfacing some extra user configuration

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.