dolt diff -r sql skips data diff when commit also has schema changes
- Dominant language
- Go
- Stars
- 24.5k
- Forks
- 873
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 120
Description
## Summary
When a commit contains both schema changes (e.g. `ALTER TABLE … ADD COLUMN`) and data changes on the same table, `dolt diff -r sql` emits **only** the schema part. The row-level deltas are reported as skipped:
```
Incompatible schema change, skipping data diff for table 't'
```
The deltas themselves are not lost — `dolt diff --stat` reports them, and the per-row data is queryable via the `dolt_diff_` system table — but the SQL emitter doesn't use either path. This breaks any tool that relies on `-r sql` for a complete replay of a commit's effect.
## Repro
```bash
mkdir /tmp/repro && cd /tmp/repro
dolt init --name t --email t@t.t
dolt sql -q "CREATE TABLE t (id INT PRIMARY KEY, name TEXT)"
dolt sql -q "INSERT INTO t VALUES (1,'a'),(2,'b'),(3,'c'),(4,'d'),(5,'e')"
dolt add -A && dolt commit -m "schema + 5 rows"
# Combined commit: 1 ALTER + 3 DELETE + 2 INSERT
dolt sql -q "ALTER TABLE t ADD COLUMN extra TEXT"
dolt sql -q "DELETE FROM t WHERE id IN (3,4,5)"
dolt sql -q "INSERT INTO t VALUES (6,'f',NULL),(7,'g',NULL)"
dolt add -A && dolt commit -m "ALTER + DELETE + INSERT"
dolt diff --stat HEAD~1 HEAD
# 2 Rows Added, 3 Rows Deleted ← deltas exist
dolt diff -r sql HEAD~1 HEAD
# ALTER TABLE `t` ADD `extra` text;
# Incompatible schema change, skipping data diff for table 't'
# ↑ DELETEs and INSERTs not emitted
# A pure-data commit on the same table works fine:
dolt sql -q "DELETE FROM t WHERE id=1; INSERT INTO t VALUES (8,'h',NULL)"
dolt add -A && dolt commit -m "data only"
dolt diff -r sql HEAD~1 HEAD
# DELETE FROM `t` WHERE `id`=1;
# INSERT INTO `t` (`id`,`name`,`extra`) VALUES (8,'h',NULL);
```
## Real-world impact
Building a tool ([dolt-replay](https://github.com/lapingvino/dolt-replay)) that replays a Dolt commit history into doltlite by walking commits and applying `dolt diff -r sql` per commit. On the bahaiwritings repo (780 commits), the replay ends up with `prayer_book_structure` ~5,000 rows over source — the first "Rebuild prayer_book_structure" commit added two columns AND deleted 5,123 rows AND inserted 2,325 — `-r sql` emitted only the two ALTERs, so the deletes never replayed.
## Suggested fixes (in order of effort)
1. **Emit the data diff against the new schema.** The `--stat` and `dolt_diff_` paths already handle the schema mismatch by projecting onto the new column set; the SQL emitter could do the same. Removed rows would be `DELETE FROM t WHERE pk = …` (no schema dependency); added/modified rows would use the new column list.
2. **`--include-data-on-schema-change` flag** that opts into best-effort data emission. Default to current behavior so this isn't a breaking change.
3. **Document the workaround** prominently in `dolt diff --help` and the docs: when `-r sql` skips a table, query `dolt_diff_` directly to recover the deltas.
## Workaround we're implementing
For each commit that contains an ALTER, additionally query `dolt_diff_` and synthesize the missing DELETE/INSERT statements ourselves. Doable but redundant — the deltas are already there, just hidden from `-r sql`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.