dolthub / dolthub/doltlite

VACUUM INTO ':memory:' is refused; copy does not inherit pager sync flags

Open
#2,407 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
268
Forks
18
Avg merge
2h 27m
Merged PRs (30d)
447

Description

Follow-up to #2373, which is closed: PR #2378 made in-transaction `VACUUM` error like stock and implemented file-backed `VACUUM INTO`. The seven `vacuum-into` testfixture gates that issue still justified are in place and still needed, so per `check_testfixture_inventory_issues_alive.sh` they need a live issue that describes the limitation as it stands now. Filed from #2400.

## Remaining gaps after #2378

`VACUUM INTO` on a DoltLite-format source writes a compacted chunk-store copy to a **file**. Two SQLite behaviors are still missing:

### 1. `VACUUM INTO ':memory:'` (and expressions that resolve to it)

Stock attaches the destination, copies pages, then detaches. `:memory:` is a valid dest. DoltLite refuses it so the VFS does not create a literal file named `:memory:`:

```
cannot VACUUM a doltlite database INTO an in-memory target
```

Verified 2026-08-24 on `build/doltlite` at `d2c52f2953`:

```sql
CREATE TABLE t(id INTEGER PRIMARY KEY, v INT);
INSERT INTO t VALUES(1,42);
VACUUM INTO ':memory:';
-- Error: cannot VACUUM a doltlite database INTO an in-memory target
```

The same error fires when the dest is an expression (`VACUUM INTO (SELECT name FROM t2)` with `name = ':memory:'`). File-backed `VACUUM INTO 'copy.db'` still works.

Gated as `class=unsupported`: `vacuum-into-200`, `vacuum-into-300`.

Making this work would mean copying the compacted image into a throwaway in-memory chunk store (stock discards the dest on DETACH anyway). The copy path today is file-only (`gcWriteCompactedTo`).

### 2. Destination does not inherit pager sync flags

Stock `VACUUM INTO` copies the source pager's `synchronous` / `fullfsync` flags onto the dest pager, so `testvfs` xSync counts match `PRAGMA synchronous`. DoltLite's copy has no pager to inherit from; dest writes go through the chunk-store file path. The dest therefore does not round-trip:

- `PRAGMA synchronous = normal|full|off|extra` (vacuum-into-710.2 .. 740.2)
- `PRAGMA fullfsync = 1` with `synchronous = full` (vacuum-into-750.2)

Gated as `class=intentional`. Closing this would need dest writes to honor those PRAGMAs via VFS xSync the way the pager does — not a missing copy of user data.

## Why the exceptions stand

Neither gap is a silent data-loss bug in file-backed `VACUUM INTO`. They are SQLite pager/memdb contracts the chunk-store copy does not have. The tests still fail on master; dropping the gates would redden `vacuum-into`.

## Work this issue tracks

- Optional: copy into an in-memory chunk store so `VACUUM INTO ':memory:'` returns `SQLITE_OK` like stock, then drop the two unsupported gates and lower the ratchet.
- Leave the five sync-count assertions gated unless dest writes start inheriting pager sync flags.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the file-only copy path at gcWriteCompactedTo and the vacuum-into-200 and vacuum-into-300 testfixture gates. Review the remaining synchronous and fullfsync assertions in vacuum-into-710.2 through 750.2 and compare their expected VFS xSync behavior. Done means either supporting an in-memory destination and removing its gates, or documenting that limitation while preserving the intentional sync-count gates.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, sqlite
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.