etcd-io / etcd-io/bbolt

Refactor freelist management & add more dedicated tests

Open
#789 9 comments 5 reactions 0 assignees View on GitHub
area/freelist stage/tracked stale
Dominant language
Go
Stars
9.7k
Forks
753
Avg merge
1d 8h
Merged PRs (30d)
18

Description

## Backgroud
There are some long standing data corruption issues, which indicate that there might be potential bug(s) in freelist management.
- https://github.com/etcd-io/bbolt/issues/778
- https://github.com/etcd-io/bbolt/issues/402
- https://github.com/etcd-io/bbolt/issues/705
- ...

I am not satisfied with the bbolt freelist management for a long time. It's hard to understand and also tightly coupled with bbolt. I have been thinking to refactor it to improve the understandability & testability.

## Refactor
The high level idea is to
- simplify the implementation to improve **understandability**;
- and introduce interface and decouple it with the bbolt TXN workflow to improve **testability**.

What we have done and are going to do:
- [x] https://github.com/etcd-io/bbolt/pull/777
- [x] https://github.com/etcd-io/bbolt/pull/783
- [x] https://github.com/etcd-io/bbolt/pull/775
- [x] https://github.com/etcd-io/bbolt/pull/788
- [x] https://github.com/etcd-io/bbolt/pull/792
- [ ] Simplify & update the logic related to the pending released free pages, refer to [doc](https://docs.google.com/spreadsheets/d/1T7ORCbv3mguq1hQoZkUg0XEzuUXFrNFFj1sUfQysZ2k/edit?gid=0#gid=0)

We also need to continue to refactor & simplify the interface from user (bbolt) perspective, in other words, we should have a clear understanding on how the interface will & should be used by bbolt. The motivation is to improve testablity.

The freelist management is the most sensitive & important part. So let's do it step by step.

## Test
Any unit tests are welcome.

But more importantly, we should add dedicated **randomized** test case to simulate **concurrent** (multiple) read TXNs and (single) writing TXN.
- We need to record all the requests or operations sent to the freelist module;
- We need to set up a list of expected behavior or invariable properties, and verify all the invariable properties are not broken during the test.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.