deven96 / deven96/ahnlich

Better persistence and compaction for Ahnlich-DB to override #9

Open
#255 0 comments 0 reactions 0 assignees View on GitHub
enhancement experimentation help wanted
Dominant language
Rust
Stars
244
Forks
29
Avg merge
2d 41m
Merged PRs (30d)
19

Description

Persistence is a second class citizen for in-memory databases but still I hate our current model of persistence (it just serves the quick and dirty I built it for initally). The obvious issues are
- We reserialize the entire database at fixed intervals
- The snapshot size grows linearly with the store size
- Due to the first reason, as the store size grows, we end up spending a longer time to be able to serialize the entire snapshot

If we are going to go for a more popular concept of an Append Only File, we could shave down some of these annoyances. We log every mutating command e.g (CREATE, SET, DELETE, DROP) in order of occurence and then we can "replay" these on startup of the database. This would also let us do things like periodic compaction and we can easil

The first issue that sticks out in my mind to solve would be `order of occurence`... given that we use a lock-free concurrent data structure (papaya) underneath, there doesn't seem to be an ideal view of the "order" of events in which an operation happened

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.