tursodatabase / tursodatabase/libsql
If bottomless misses uploading a single frame, then restoration will fail
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 17.2k
- Forks
- 531
- Avg merge
- 1h 12m
- Merged PRs (30d)
- 1
Description
Lets say there is an network issue and bottomless failed to upload a single frame, then sent rest of the frames, then restoration can fail. Snapshot at shutdown fixes this issue, but if the DB did not have a clean shutdown then it won't work.
To reproduce this issue (you can use Nemesis):
- Run sqld and insert a few rows in a db
- Stop local minio server
- Insert a few more rows
- Resume minio and insert few more rows
- Kill the sqld (i.e. unclean shutdown)
- On restart, the restoration would fail
Possible solution:
We write all the frames to disk before uploading to S3. We could probably build a local queue and when an upload is successful, it should check if there are any previously unsent frames present on disk and upload them.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the bottomless frame upload, local-disk queue, and restoration paths; the issue does not name specific files or tests. Reproduce the failure with the listed MinIO outage and unclean shutdown steps, then verify that a missed frame is uploaded later and restoration succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- cloud, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100