etcd-io / etcd-io/bbolt

Use sparse files (FALLOC_FL_PUNCH_HOLE) to deallocate unused pages on disk

Open
#1,063 2 comments 0 reactions 1 assignee Claimed by @fuweid View on GitHub
stage/tracked
Dominant language
Go
Stars
9.7k
Forks
753
Avg merge
1d 8h
Merged PRs (30d)
18

Description

The README currently says:

https://github.com/etcd-io/bbolt/blob/dbe1e6dc0d30fc1ae4c7c8078615dd2bbccf4499/README.md#L886-L891

The missing ability to truncate to return space to the filesystem could be mitigated, at least on ext4/xfs/btrfs on linux, by having bbolt call

```
unix.Fallocate(fd, unix.FALLOC_FL_PUNCH_HOLE|unix.FALLOC_FL_KEEP_SIZE, offset, length)
```

either eagerly whenever a page is added on the freelist or lazily later during a background operation, with `offset` the offset of the page in bytes from the start of the file, and `length` the size of the page in bytes.

This would instruct the kernel/fs to free the space on disk corresponding to that page, thereby making it available again to the rest of the system.tio

Given that the actual effectiveness of this approach depends on the combination of (at least) OS, OS version, filesystem, filesystem page size, database page size, and physical disk page size, whether to perform the Fallocate call should be controlled by a database option, defaulting to false.

Note: this is the same as https://github.com/etcd-io/bbolt/issues/125, that was automatically closed as stale due to no activity.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.