tursodatabase / tursodatabase/libsql
Batch pushed WAL frames
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 17.2k
- Forks
- 531
- Avg merge
- 1h 12m
- Merged PRs (30d)
- 1
Description
We currently send one frame at a time, which has lots of overhead from TLS handshake, for example. Let's send WAL frames in batches -- perhaps make it configurable and default to 32 KiB (which is 8 frames). We could also make batching smart and send full transactions at at time (or multiple transactions). The server side is built to allow partial transactions to be sent, but sending full transactions has less overhead.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are identified. Start by locating the WAL frame sending path and the server-side handling for partial transactions; done means configurable batching, a 32 KiB default, and support for sending complete transactions or multiple transactions where appropriate.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, sqlite
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100