googleapis / googleapis/google-cloud-python

Implement manual flush in the data client's MutationsBatcher

Aperta
#15,295 1 commento 0 reazioni 1 assegnatario Rivendicata da @gkevinzheng Vedi su GitHub
api: bigtable priority: p3 type: feature request
Lingua principale
Python
Stelle
5.4k
Fork
1.8k
Merge medio
3g 4h
PR unite (30g)
122

Descrizione

The [MutationsBatcher class](https://github.com/googleapis/python-bigtable/blob/3648d7000df4a108524e29a9c55639dc92c1c2cc/google/cloud/bigtable/data/_async/mutations_batcher.py#L182) lets users add mutations to the batcher object, which then periodically flushes in batches. It currently flushed based on time, mutation count, mutation size, or when the batcher is closed. It doesn't expose a way for users to call flush manually

We'd need to close this feature gap to support the legacy client. It should match the [existing function signature](https://github.com/googleapis/python-bigtable/blob/7cab3e88b7c7e880d7ecaa574f70a88fe35c700f/google/cloud/bigtable/batcher.py#L289), meaning it should block until completion, and raise any exceptions for failed batches immediatly

We had a [partial implementation staged](https://github.com/googleapis/python-bigtable/pull/770/commits/6da9a64b2f95bacade5ee9483c2025a16fc3e7dd) when the batcher was originally written, but we pulled it out to keep the flushing flow simple. We can re-use some of that work here

If getting async and sync to work is difficult, we could consider implementing this only for the sync client. But ideally, we should have the functionality in both

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.