dolthub / dolthub/doltlite

Stateful VC fuzzer generates one table shape: no non-integer PKs, generated columns, DROP/RENAME COLUMN, or history rewriting

Closed
#2,913 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
268
Forks
18
Avg merge
2h 27m
Merged PRs (30d)
447

Description

Found in a full-repo review at `0ba280f06f`.

The stateful VC fuzzer's oracle is strong; its schema universe is one table shape. Every recurring bug cluster in the tracker sits in the region it cannot generate.

## What it checks (good)

`test/vc_stateful_fuzzer.py` verifies, per step: a full row model and HEAD model per branch (`assert_rows`, `:169`), a branch/tag ref model (`assert_refs`, `:205`), `dolt_hashof_table` / `dolt_hashof_db` shape and stability (`:186`, `:198`), `PRAGMA integrity_check` (`:243`), views still queryable (`:563`), triggers still fire (`:577`), fts5 shadow-table consistency (`:395`), and REINDEX-preserves-answers via `dolt_hashof_index` (`:437`). That is a genuinely strong invariant set, and it is why the fuzzer has found real bugs (#2731, #2446, #2405).

## What it generates (narrow)

- Every generated table is `id INTEGER PRIMARY KEY, payload TEXT` — `:667`, `:891`, `:1221`.
- The only column-level DDL is `ALTER TABLE ADD COLUMN c TEXT` — `:893`. No `DEFAULT`, no `NOT NULL`, no `DROP COLUMN`, no `RENAME COLUMN`.
- Zero occurrences of `STORED`, `VIRTUAL` or `GENERATED` in the whole file.
- No TEXT, composite, or `WITHOUT ROWID` primary keys.
- No `UNIQUE` / `CHECK` / `FOREIGN KEY` added mid-run.
- Operators cover merge, branch, tag, push/pull and revert — but not `dolt_cherry_pick` or `dolt_rebase`.
- No explicit-transaction interleaving (`BEGIN` … `dolt_*` … `ROLLBACK`).

## Why this is the highest-leverage gap

Of 192 issues filed since #2380, about 20 came from CI or nightly runs, and only ~5 of those were engine bugs. Discovery is overwhelmingly human review. Meanwhile the clusters by title are: merge correctness 21, name/schema resolution 18, clustered-PK/rowid seam 15, remotes 11, performance 10, test floors 10, `dolt_*` transaction semantics 9, gc/durability 9 — i.e. precisely the table shapes and DDL the fuzzer never emits.

Two findings from this review are direct examples:

- The `ADD COLUMN ... DEFAULT` after `DROP COLUMN` regression needs `DROP COLUMN` and a `DEFAULT` — the fuzzer emits neither.
- The cherry-pick / rebase schema-action loss needs a DDL on both sides plus a history-rewriting operator — the fuzzer emits neither.

Both are mechanically checkable with oracles the fuzzer already has.

## Suggested additions

1. PK shapes: TEXT, composite, `WITHOUT ROWID`, DESC — chosen per generated table.
2. Generated columns, STORED and VIRTUAL, including one in an index, with the self-checking property `count(*) WHERE g IS NOT () = 0` after every operation. (That property over 42 hand-written merge scenarios is clean on master today, so it would start green.)
3. `ADD COLUMN` with `DEFAULT` and `NOT NULL DEFAULT`; `DROP COLUMN`; `RENAME COLUMN`.
4. `dolt_cherry_pick` and `dolt_rebase` as operators.
5. Constraints (`UNIQUE`, `CHECK`, `FK`) added and dropped mid-run.
6. Explicit transactions wrapping VC calls, with `ROLLBACK` as an outcome. (#2152 closed a similar coverage gap; this is the schema-side equivalent.)

The broader principle, which `test/oracle_schema_change_cherry_pick_revert_test.sh` also violates by testing 12 hand-picked scenarios that all leave the target branch's schema untouched: generate the matrix, do not enumerate examples.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with test/vc_stateful_fuzzer.py, especially table generation at lines 667, 891, and 1221, column DDL at line 893, and the existing assertions around lines 169, 186, 198, 205, 243, 395, 437, and 563-577. Extend the generated schema and operators with the suggested PK shapes, generated columns, DDL, history rewriting, constraints, and transactions, then verify the existing oracles remain green and cover the new cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
databases, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.