Stateful VC fuzzer generates one table shape: no non-integer PKs, generated columns, DROP/RENAME COLUMN, or history rewriting
- Dominant language
- C
- Stars
- 268
- Forks
- 18
- Avg merge
- 2h 27m
- Merged PRs (30d)
- 447
Description
Found in a full-repo review at `0ba280f06f`.
The stateful VC fuzzer's oracle is strong; its schema universe is one table shape. Every recurring bug cluster in the tracker sits in the region it cannot generate.
## What it checks (good)
`test/vc_stateful_fuzzer.py` verifies, per step: a full row model and HEAD model per branch (`assert_rows`, `:169`), a branch/tag ref model (`assert_refs`, `:205`), `dolt_hashof_table` / `dolt_hashof_db` shape and stability (`:186`, `:198`), `PRAGMA integrity_check` (`:243`), views still queryable (`:563`), triggers still fire (`:577`), fts5 shadow-table consistency (`:395`), and REINDEX-preserves-answers via `dolt_hashof_index` (`:437`). That is a genuinely strong invariant set, and it is why the fuzzer has found real bugs (#2731, #2446, #2405).
## What it generates (narrow)
- Every generated table is `id INTEGER PRIMARY KEY, payload TEXT` — `:667`, `:891`, `:1221`.
- The only column-level DDL is `ALTER TABLE ADD COLUMN c TEXT` — `:893`. No `DEFAULT`, no `NOT NULL`, no `DROP COLUMN`, no `RENAME COLUMN`.
- Zero occurrences of `STORED`, `VIRTUAL` or `GENERATED` in the whole file.
- No TEXT, composite, or `WITHOUT ROWID` primary keys.
- No `UNIQUE` / `CHECK` / `FOREIGN KEY` added mid-run.
- Operators cover merge, branch, tag, push/pull and revert — but not `dolt_cherry_pick` or `dolt_rebase`.
- No explicit-transaction interleaving (`BEGIN` … `dolt_*` … `ROLLBACK`).
## Why this is the highest-leverage gap
Of 192 issues filed since #2380, about 20 came from CI or nightly runs, and only ~5 of those were engine bugs. Discovery is overwhelmingly human review. Meanwhile the clusters by title are: merge correctness 21, name/schema resolution 18, clustered-PK/rowid seam 15, remotes 11, performance 10, test floors 10, `dolt_*` transaction semantics 9, gc/durability 9 — i.e. precisely the table shapes and DDL the fuzzer never emits.
Two findings from this review are direct examples:
- The `ADD COLUMN ... DEFAULT` after `DROP COLUMN` regression needs `DROP COLUMN` and a `DEFAULT` — the fuzzer emits neither.
- The cherry-pick / rebase schema-action loss needs a DDL on both sides plus a history-rewriting operator — the fuzzer emits neither.
Both are mechanically checkable with oracles the fuzzer already has.
## Suggested additions
1. PK shapes: TEXT, composite, `WITHOUT ROWID`, DESC — chosen per generated table.
2. Generated columns, STORED and VIRTUAL, including one in an index, with the self-checking property `count(*) WHERE g IS NOT () = 0` after every operation. (That property over 42 hand-written merge scenarios is clean on master today, so it would start green.)
3. `ADD COLUMN` with `DEFAULT` and `NOT NULL DEFAULT`; `DROP COLUMN`; `RENAME COLUMN`.
4. `dolt_cherry_pick` and `dolt_rebase` as operators.
5. Constraints (`UNIQUE`, `CHECK`, `FK`) added and dropped mid-run.
6. Explicit transactions wrapping VC calls, with `ROLLBACK` as an outcome. (#2152 closed a similar coverage gap; this is the schema-side equivalent.)
The broader principle, which `test/oracle_schema_change_cherry_pick_revert_test.sh` also violates by testing 12 hand-picked scenarios that all leave the target branch's schema untouched: generate the matrix, do not enumerate examples.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with test/vc_stateful_fuzzer.py, especially table generation at lines 667, 891, and 1221, column DDL at line 893, and the existing assertions around lines 169, 186, 198, 205, 243, 395, 437, and 563-577. Extend the generated schema and operators with the suggested PK shapes, generated columns, DDL, history rewriting, constraints, and transactions, then verify the existing oracles remain green and cover the new cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sqlite
- Domain
- databases, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100