apple / apple/foundationdb

SQLite files are artificially limited to around 4TB but up to 16TB is supported (with 4k pages)

Open
#3,736 0 comments 0 reactions 1 assignee Claimed by @satherton View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

There should be a server knob, probably `SQLITE_MAX_FILE_SIZE` which is used to calculate and set the SQLite max page count during initialization. Another option would be `SQLITE_MAX_PAGE_COUNT` which more directly maps to the SQLite option but it's more awkward to reason about since people usually think in terms of disk space.

I think it would be fine to use SQLite's max supported file size with 4k pages, `4096 * (2^32 - 2)`, which is about 16TB. The knob's existence allows it to be lowered in case a user wants to limit usage of a disk volume to some size.

Also, in `VFSAsync.cpp` this new knob should be obeyed when increasing file size.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.