fluvio-community / fluvio-community/sql-connector

Batch Inserts and Upserts

Open
#154 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
8
Forks
4
PR merge metrics
No merged PRs in 30d

Description

I would like to have the posibility to perform batch inserts.
Right now every record gets inserted/upserted into the database one by one, this can be a performance bottleneck if too many records are produced.
Its basically the same as a SQL query inside a loop.

A batch inserter mechanism could collect up to N records and also wait up to X seconds before sending the query to the database anyway. Whichever case happens first, so there is not a batch "forever" in the waiting to be sent to the DB, it gets flushed at least at some point.

I'll provide a example implementation.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.