`checkpoint --verify` cannot distinguish a failed restore from a non-deterministic checkpoint
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 0
- Forks
- 0
- Avg merge
- 1h 58m
- Merged PRs (30d)
- 48
Description
Summary
newgit checkpoint --verify compares two state refs and reports MISMATCH
when they differ. It has both deposited snapshots in hand and does not look at
them, so it cannot distinguish the failure it exists to catch (a restore that
landed on the wrong state) from a checkpoint command that simply does not
produce stable bytes.
The output is two opaque revs, which is not enough to tell which one you have.
What happened
Verifying a Postgres data-dump resource:
resource: db MISMATCH before: tracker:db-snapshots@667c1462a291 after: tracker:db-snapshots@1fd633af04cd
NOT verified: `feature/ng-smoke`'s restore either failed or landed on a
different state ref.
The restore was correct. pg_dump emits a \restrict <token> / \unrestrict <token> pair at the top and bottom of every dump with a freshly generated
random token each run, so two dumps of a byte-identical database hash
differently.
I found this by locating the snapshots under .newgit/snapshots/db-snapshots/
by hand and diffing them:
$ diff before/db.sql after/db.sql | wc -l
4 # 2 lines, both the \restrict preamble
$ wc -l before/db.sql
3299
Two lines out of 3299, both obviously a nonce. That is instantly diagnostic,
and newgit was holding both files the whole time.
Why this matters
--verify is one of the best ideas in the tool — "a restore can exit clean and
still land on the wrong state" is exactly right, and the — restore never exercised flag caught a genuinely broken [restore] for me earlier in the
same integration. But a check that cries wolf on nondeterministic output
teaches you to disregard it, which costs you the real finding later.
The distinction is also actionable in opposite directions:
- restore landed wrong — fix the
[restore]command - checkpoint is nondeterministic — fix the
[checkpoint]command
Nothing in the current output points at either.
Suggested fix
On a mismatch, diff the two deposited snapshots and say what was found
(crates/newgit/src/main.rs:1308, :1320):
resource: db MISMATCH before: tracker:db-snapshots@667c14 after: tracker:db-snapshots@1fd633
deposited content differs on 2 of 3299 lines — this looks like a
non-deterministic [checkpoint] command, not a failed restore:
- \restrict eZYVhzvJBP8npSPM99dImZ5VGvNTb4FLoETskEcsi7IJMDZVIBfm
+ \restrict mQ2kTnR7dFsX1pLvWcYbZaHgEuJoNi5rCtA9SdKfMhBxUyPq3Ge
Identical content with differing refs is a strong, cheap signal: if the bytes
match, the restore demonstrably worked and only the ref derivation is unstable.
Worth a line in the reference too, under [checkpoint]: a command
checkpoint should produce the same bytes for the same state, and common dump
tools do not by default. I only learned this by hitting it.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The mismatch handling is in crates/newgit/src/main.rs around lines 1308 and 1320; start there and inspect how the deposited snapshots are available. Compare the two snapshots on mismatch and report whether their content is identical or differs in a small, diagnostic way, so --verify distinguishes a failed restore from nondeterministic checkpoint output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, rust
- Domain
- cli, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100