googleapis / googleapis/google-cloud-python

Implement manual flush in the data client's MutationsBatcher

Offen
#15,295 1 Kommentar 0 Reaktionen 1 zugewiesene Person Beansprucht von @gkevinzheng Auf GitHub ansehen
api: bigtable priority: p3 type: feature request
Vorherrschende Sprache
Python
Sterne
5.4k
Forks
1.8k
Ø Merge
3 T. 4 Std.
Gemergte PRs (30 T.)
122

Beschreibung

The [MutationsBatcher class](https://github.com/googleapis/python-bigtable/blob/3648d7000df4a108524e29a9c55639dc92c1c2cc/google/cloud/bigtable/data/_async/mutations_batcher.py#L182) lets users add mutations to the batcher object, which then periodically flushes in batches. It currently flushed based on time, mutation count, mutation size, or when the batcher is closed. It doesn't expose a way for users to call flush manually

We'd need to close this feature gap to support the legacy client. It should match the [existing function signature](https://github.com/googleapis/python-bigtable/blob/7cab3e88b7c7e880d7ecaa574f70a88fe35c700f/google/cloud/bigtable/batcher.py#L289), meaning it should block until completion, and raise any exceptions for failed batches immediatly

We had a [partial implementation staged](https://github.com/googleapis/python-bigtable/pull/770/commits/6da9a64b2f95bacade5ee9483c2025a16fc3e7dd) when the batcher was originally written, but we pulled it out to keep the flushing flow simple. We can re-use some of that work here

If getting async and sync to work is difficult, we could consider implementing this only for the sync client. But ideally, we should have the functionality in both

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.