sqlite_dbpage: synthesized page 1 only; deeper pages reject as unsupported
- Dominant language
- C
- Stars
- 268
- Forks
- 18
- Avg merge
- 2h 26m
- Merged PRs (30d)
- 454
Description
## Summary
DoltLite exposes \`sqlite_dbpage\` for tools that expect it, but the implementation diverges from stock SQLite. Page 1 returns a synthesized header populated with derived values; any other \`pgno\` now errors instead of returning EOF (the prior behavior silently returned nothing).
## Why it diverges
Stock SQLite's database file is a linear array of fixed-size pages (default 4 KiB). \`sqlite_dbpage\` is a thin SQL accessor over that array — \`pgno=N\` returns bytes \`(N-1)*pageSize .. N*pageSize\`.
DoltLite has no pages. Storage is a tree of content-addressed chunks; chunks are variable-sized and addressed by hash, not by sequential page number. There is no meaningful \`pgno=47\`.
## What's implemented (src/doltlite_dbpage.c)
- \`pgno=1\` (or full scan): synthesizes a 4 KiB page-1 header with realistic-looking fields:
- \`SQLite format 3\` magic
- \`pageSize = 4096\`
- \`changeCounter\` derived from the session head commit hash
- \`pageCount = user-table-count\` (deliberately not the chunk count)
- \`schemaCookie\` derived from the head catalog hash
- \`largestRootPgno\` = max(table iTable)
- \`pgno > 1\`: \`SQLITE_ERROR\` with message \"doltlite: sqlite_dbpage only supports pgno=1 (content-addressed chunk store has no page layout)\".
## Impact on tools
- \`sqlite3_analyzer\`, page-utilization scripts, raw page-level backup tools: do not work. They expect to walk the full page range and will get an error at the first \`pgno > 1\`.
- Header-only inspection (e.g. detect SQLite format version, page size, schema cookie): works against page 1.
- Anything that scans \`sqlite_dbpage\` for diagnostics: gets a single row.
## Why the error now (was EOF before)
Silently returning zero rows for \`pgno > 1\` made tools think every page past page 1 was missing or empty, which produced misleading "database has 1 page" reports without any signal that doltlite couldn't answer. The error makes the unsupportedness explicit.
## Workarounds
- For database-size monitoring: use \`SELECT count(*)\` on user tables or query the chunk store via doltlite-specific functions instead.
- For page-level forensics: this isn't a workflow doltlite supports.
## Future options
1. Stay where we are. \`sqlite_dbpage\` covers \"is this a SQLite-format file? what's the schema cookie?\" and explicitly rejects the rest.
2. Build a chunk-store-flavored vtable (e.g. \`doltlite_chunks(hash, size, data)\`) for the diagnostic use cases. Doesn't pretend to be \`sqlite_dbpage\`.
3. Synthesize a believable page-mapping over the chunk store so \`sqlite3_analyzer\` reports something. Most work, dubious payoff — the numbers would be lies aligned to look real.
Filing here so the divergence is visible to anyone who finds doltlite via the standard SQLite tooling.
## Related
- Deep-review finding **P8** (the substance of the divergence).
- Deep-review finding **P9** (the registration plumbing): fixed in the same PR that filed this issue.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with src/doltlite_dbpage.c and review the referenced P8 and P9 findings, including the related registration changes. The issue documents current behavior and lists three possible future directions, but does not select an implementation target; completion criteria therefore need to be agreed before coding can begin.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, sqlite
- Domain
- databases
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100