dolthub / dolthub/doltlite

corrupt-3.3 is an intermittent divergence and trips the exception ratchet

Open
#1,963 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
268
Forks
18
Avg merge
2h 27m
Merged PRs (30d)
447

Description

`corrupt corrupt-3.3` is pinned in `test/known_testfixture_divergences.txt`, but its outcome varies between runs, so `sqlite-regression-storage` intermittently fails with:

```
SQLite regression storage: 1 entry/entries should be updated or removed
stale/mismatched entry list:
corrupt corrupt-3.3
```

Demonstrated on #1958: the bucket failed, and a re-run of the **same commit and same job** passed. #1954, on the same master base, passed it. Same platform, same `TESTFIXTURE_VARIANT: coverage`.

The assertion is not a raw byte poke, so `93c481b6c2`'s reasoning does not directly apply: `corrupt-3.2` swaps the `rootpage` values of `t1` and `t1i1` through `writable_schema`, and 3.3 expects `INSERT INTO t1` to report "database disk image is malformed". In doltlite `rootpage` is an iTable number into the prolly catalog, so this is really "is reading a table through an index's tree detected".

Hypothesis for the variability, not yet proven: `doltlite_core.c` stamps commits with `time(0)`, so the fixture's content hashes differ per run, chunk layout differs with them, and whether the mismatched-tree read trips a validation check follows. If that is right, giving the suites a fixed clock would remove a whole class of content-hash variation rather than just this entry. The gate's current comment ("hexio lands on CTLD bytes") describes its 3.4/3.5 siblings, not 3.3.

Either way the entry cannot be pinned as-is: a flaky assertion trips the ratchet in the "should be removed" direction and produces reds on unrelated PRs, which erodes the property that a red non-perf job is a real signal.

Options: make test commits deterministic so the outcome stabilises; or add an explicit "unstable" disposition that suppresses both ratchet directions for such an entry.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the sqlite-regression-storage failure for corrupt-3.3 and inspect test/known_testfixture_divergences.txt alongside doltlite_core.c. Verify whether time(0)-based commit content changes the validation outcome, then choose and implement a stabilization or explicit unstable disposition so the ratchet no longer reports intermittent reds.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, sqlite
Domain
databases, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.