Pluggable checkpoint stores: abstraction refactor (tracking)

Goal

Make it cheap to add and experiment with new checkpoint storage backends. Adding a store should mean: implement read + write (+ push/fetch if it needs a remote round-trip), register it, and select it via settings — without touching the ~15 construction sites or reimplementing git-specific machinery.

This is the runtime half that #1051 (settings schema v2: checkpoints.{primary,mirrors,git}) was missing — that PR modeled the right config shape but nothing consumes it yet.

Findings driving the design

1. One fat interface, one implementation, scattered construction. Store (cmd/entire/cli/checkpoint/checkpoint.go:70) is implemented only by GitStore, built via NewGitStore(repo, ResolveCommittedRefs(ctx)) duplicated across ~15 sites in cli and strategy.

2. The interface conflates two unrelated halves:

   • Temporary/shadow (WriteTemporary*, ListTemporaryCheckpoints) — per-step working-tree snapshots on git shadow branches. Inherently git-only.
   • Committed (ReadCommitted, WriteCommitted, UpdateCommitted, ListCommitted, ReadSessionContent*) — the permanent record + reads. The pluggable part.

3. Push/fetch — the "update endpoint" — isn't in the store at all. It lives in strategy (checkpoint_remote.go, manual_commit_push.go, push_common.go, FetchMetadataBranch), hardcoded to git refs. A new store can't carry its own sync.

4. No store-type seam exists. The only variation point, CommittedRefs (v1 vs v1.1 mirror), is just different ref names on the same git backend.

5. Rewind is deprecated, not removed (shipped in today's release); removal will take time. Rewind has two modes, and only one is git/temporary-only:

   • Working-tree rewind — restores files from shadow-branch commits. Temporary/git-only.
   • Logs-only rewind — GetRewindPoints builds logs-only entries from committed checkpoint metadata, and RestoreLogsOnly reads the committed summary + session content from entire/checkpoints/v1 (manual_commit_rewind.go:158,646,688). This DOES touch committed storage — i.e. the pluggable surface.

   Resume/attach/status read committed + session state + on-disk transcripts; shadow branches are local-only, never pushed, deleted after condensation. Shadow branches' one irreplaceable capability (intermediate working-tree content) was consumed only by working-tree rewind — the committed record never stored it. Condensation's read of transcript/prompts from the shadow tree is a fallback; that data lives on disk.

The abstraction

Split the fat interface; make only the committed half pluggable. Keep the temporary half as a single git implementation that rewind continues to ride.

Build on the existing committed interfaces, do not invent colliding names. CommittedReader and CommittedListReader already exist (committed_reader_resolve.go:11,17) and define real read semantics. The pluggable surface must extend those, and Phase 1 must start with a complete method inventory — not the abbreviated sketch below. Known live methods beyond ReadCommitted/ReadSessionContent/ListCommitted that callers depend on and that must be covered (or callers will still need a concrete *GitStore):

• ReadSessionContentByID
• ReadSessionMetadata, ReadSessionMetadataAndPrompts, ReadSessionPrompts
• WriteCommitted, UpdateCommitted
• UpdateSummary, UpdateCheckpointSummary
• GetCheckpointAuthor

(refs: review_context.go:32, explain.go:931, manual_commit_hooks.go:1157)

// CommittedWriter is the FULL write surface — this is what mirror fan-out targets,
// so it must include every write-like op, not just WriteCommitted/UpdateCommitted.
// Summary generation and combined attribution write metadata then mirror separately today
// (explain.go:931, manual_commit_hooks.go:1157, manual_commit_condensation.go:287).
type CommittedWriter interface {
    WriteCommitted(...) ; UpdateCommitted(...)
    UpdateSummary(...) ; UpdateCheckpointSummary(...)
}

// Pluggable committed surface. Extends the EXISTING CommittedReader / CommittedListReader
// (committed_reader_resolve.go) — final method set comes from the Phase 1 inventory above.
type CommittedStore interface {
    CommittedListReader   // existing: ReadCommitted, ReadSessionContent, ListCommitted,
                          // ReadSessionMetadata, ReadSessionPrompts
    ReadSessionContentByID(...) ; ReadSessionMetadataAndPrompts(...)
    CommittedWriter
}

// AuthorReader is an OPTIONAL capability — GetCheckpointAuthor is git-history-specific
// (commit author of the metadata commit) and already optional/best-effort in explain
// (explain.go:722, committed.go:2275). Model it as a capability rather than forcing every
// backend to implement it; non-git stores simply don't advertise it (callers already
// tolerate an empty author).
// (returns the existing Author{Name, Email} struct — explain uses both name and email,
// committed.go:2266,2275, explain.go:722 — NOT a bare string)
type AuthorReader interface { GetCheckpointAuthor(ctx, id.CheckpointID) (Author, error) }

// Optional sync capability — see Phase 3 for the real operation set; NOT a 2-method Push/Fetch.
type Syncable interface { /* explicit sync operations + options, defined in Phase 3 */ }

// Primary + mirrors topology (mirrors PR #1051's settings shape).
// NOTE: Mirrors here = INDEPENDENT BACKEND mirroring only (see "Two kinds of mirror" below).
// v1.1 ref-mirroring is NOT modeled here — it stays internal to the git store.
type CheckpointStores struct {
    Primary CommittedStore     // source of truth; serves ALL reads
    Mirrors []CommittedWriter  // independent backends: each gets its OWN WriteCommitted call, best-effort, failures logged
}

// Registry/factory keyed by backend type. OpenOptions carries the CLI-level BlobFetchFunc
// AND explicit committed-ref / injected-settings overrides (see below).
type OpenOptions struct {
    BlobFetcher BlobFetchFunc // injected by CLI; factory in checkpoint cannot know it otherwise

    // Committed-ref override. MUST be explicit, not "read current settings". attach injects
    // settings and applies refs.PrimaryAsRead() guards (attach.go:62,366,466); Phase 0 must
    // preserve that exact topology, NOT silently switch attach to live-settings resolution.
    Settings *settings.EntireSettings // nil → resolve from disk
    Refs     *CommittedRefs            // nil → resolve from Settings; non-nil wins (e.g. PrimaryAsRead)
}
func Register(typ string, f func(ctx, repo, cfg) (CommittedStore, error))
func Open(ctx, repo, OpenOptions) (*CheckpointStores, error) // reads settings → builds Primary + Mirrors

Non-read/write accessors must be in the facade plan

Callers don't only call read/write methods — they reach for store.Refs() and store.Repository() for git-topology decisions:

• resume uses store.Refs() to decide remote bootstrap / origin-check behavior (resume.go:193).
• Strategy uses store.Refs() for v1.1 mirror advancement (manual_commit_condensation.go:287, manual_commit_hooks.go:2821).
• explain uses store.Repository() for mirror repair after summary generation (explain.go:936).

Phase 0 must inventory these alongside the read/write sites, and the facade must expose them so callers do not type-assert back to *GitStore. Two acceptable shapes:

• Higher-level operations on the facade that encapsulate the ref/repo logic (preferred long-term — e.g. a RepairMirror() / BootstrapFromOrigin() method), or
• Resolved-topology accessors during the transition: the facade exposes the resolved CommittedRefs and (for the git backend) the repo, so existing call sites keep working before Phase 3 moves the logic inside.

These accessors are inherently git-shaped; long-term they belong behind sync/admin capabilities (Phase 3), but Phase 0 must not strand them on the concrete type.

Phase 0 returns the FINAL facade shape (avoid a second call-site migration)

If Phase 0 returns *GitStore but Phase 2 flips Open to return *CheckpointStores, every call site migrates twice. Land the final signature in Phase 0: Open(ctx, repo, OpenOptions) (*CheckpointStores, error) from day one. Until Phase 1 splits the interface, CheckpointStores.Primary simply holds the concrete *GitStore and the facade also exposes the git-only temporary capability (e.g. a Temporary() *GitStore accessor) so rewind/explain keep working. Phases 1–2 then change internals only — no further call-site churn.

Two independent selection axes:

• Temporary capture is always git shadow branches (only thing that can do it). Working-tree rewind is therefore a git-backend capability — if a non-git primary is selected, it isn't offered (fine — it's deprecated).
• Committed read/write/sync routes to the selected backend.

Rewind capability decision (must be made): Because logs-only rewind reads committed storage (the pluggable surface), it could in principle work against any CommittedStore, while working-tree rewind needs the git temporary store. So rewind is not cleanly "git-only." Decide one of:

• (a) Recommended — disable rewind entirely on non-git primaries. Simplest; rewind is deprecated and heading for removal, so not worth wiring logs-only through new backends.
• (b) Support logs-only rewind on any CommittedStore, working-tree rewind only on git.

Either way, with today's default (git primary) rewind behaves exactly as now.

Condensation becomes the transform across the seam: reads transcript/prompts from the git shadow tree (unchanged), writes through the pluggable committed writer. Push/fetch moves out of strategy into the git store's sync impl (Phase 3).

BlobFetchFunc injection (resolved)

Reads after treeless/filtered fetches depend on SetBlobFetcher(FetchBlobsByHash) wired at the CLI layer (resume.go:191, explain.go:861, manual_commit.go:45). A factory inside checkpoint cannot know that CLI-level fetcher, so Open must take OpenOptions{ BlobFetcher } rather than being a bare Open(ctx, repo).

Two kinds of mirror — keep them distinct

Mirrors []CommittedWriter would change v1.1 semantics if applied to it. There are two fundamentally different mirror concepts, and only one is the cross-backend fan-out:

1. Same-git-object ref mirroring (v1.1, existing). GitStore.WriteCommitted advances only the primary ref; then MirrorCommittedMetadataRef points the v1.1 read ref at the exact same commit hash (committed.go:122, v1_custom_ref_mirror.go:152). There is no second write and no second commit history — one object, two refs. This must stay internal to the git store and must NOT be expressed as a CommittedWriter in Mirrors[], or v1.1 would gain a divergent second history.
2. Independent backend mirroring (new, the fan-out). A separate store (e.g. gmeta/S3) receives its own WriteCommitted call and produces its own objects. This is what CheckpointStores.Mirrors models — best-effort, failures logged.

The issue's "mirrors are write-only fan-out" framing refers to #2 only.

Legacy v1.1 must NOT change read behavior

Today checkpoints_version: "1.1" makes committed reads target refs/entire/checkpoints/v1.1 (committed_refs.go:74; docs/architecture/sessions-and-checkpoints.md:227). The "primary serves all reads, mirrors are write-only" framing is a behavior change unless we map it correctly. Requirement: legacy v1.1 maps to a git primary store whose read ref is the custom v1.1 ref (Primary writes v1 branch + mirrors to v1.1, reads resolve against v1.1). The new topology must preserve exactly this; "mirrors are write-only" is the model for new backends, not a redefinition of v1.1.

Phases

Each phase is independently shippable and links one or more PRs below.

• Phase 0 — centralize construction. Replace the ~15 scattered NewGitStore(...) calls with one checkpoint.Open(ctx, repo, OpenOptions{...}) returning the final *CheckpointStores facade (see above), threading existing BlobFetcher wiring through OpenOptions. The construction grep must include in-package constructors too — e.g. checkpoint.LookupSessionLog opens a repo and builds GitStore directly inside the checkpoint package (committed.go:1409), not just the cli/strategy call sites. Also inventory the non-read/write accessors (store.Refs(), store.Repository() — see "Non-read/write accessors" above) and surface them on the facade so no caller type-asserts back to *GitStore. Preserve every site's current ref topology exactly via OpenOptions.Settings/Refs — in particular attach injects settings and uses refs.PrimaryAsRead() (attach.go:62,366,466) and must NOT regress to live-settings resolution. Pure mechanical, no behavior change. This alone unblocks experimentation and is the highest-leverage step. — PR: TBD
• Phase 1 — split the interface. First produce the complete committed method inventory (see list above + a fresh grep of *GitStore call sites). Carve a CommittedStore extending the existing CommittedReader/CommittedListReader out of Store; leave the temporary half on a git-only TemporaryStore. Temporary inventory is broader than the earlier sketch — must include WriteTemporary, WriteTemporaryTask, ListTemporary, ListTemporaryCheckpoints, ListAllTemporaryCheckpoints, ListCheckpointsForBranch (explain's reachable shadow-branch view, explain.go:2175, temporary.go:500), GetTranscriptFromCommit, ShadowBranchExists (refs: temporary.go:588, rewind.go:745, manual_commit_git.go:58). Either cover all of these in the git-only temporary capability, or explicitly accept local *GitStore use in rewind/explain until rewind is removed. The strategy holds both — a git-pinned TemporaryStore and a resolved CommittedStore. — PR: TBD
• Phase 2 — topology + registry. Introduce CheckpointStores (primary + mirrors) and the Register/Open factory; wire PR #1051's settings schema to drive selection. Re-express v1.1 as a git primary whose read ref stays the custom ref (no read-behavior change — see above). A new backend = one self-contained file + a settings entry. — PR: #1533 (uses the git-branch type name + git-backed-primary / one-of-each-type rules; v1.1 re-expression dropped — see PR)
• Phase 3 — move sync into the store. Relocate push/fetch from strategy into the git store's sync impl. Define explicit sync operations/options, not a 2-method Push/Fetch: push needs the git remote + checkpoint-remote resolution (manual_commit_push.go:21); fetch has four modes — tree-only, full, checkpoint-remote, and blob-by-hash (git_operations.go:412,420,523). Note the package-cycle work: strategy currently owns these helpers and imports checkpoint, so moving them into checkpoint (or a new lower-level package) requires untangling that dependency. Also migrate doctor's mirror repair/diagnosis — doctor directly calls strategy.MirrorCommittedMetadataRef to repair the v1.1 read mirror (doctor.go:456) and doctor bundle calls DiagnoseCommittedMetadataMirror (doctor_bundle.go:180); if mirror behavior moves into the store, both need a store/admin capability (or a git-only repair/diagnose path) rather than reaching into strategy. — PR: TBD
• Later (gated on rewind's actual removal) — retire the temporary layer. First validate condensation can build a complete committed checkpoint from disk + session state alone across all paths (mid-turn commits, multi-session, subagent tasks — the disk fallback exists for a reason). Then delete shadow branches and their maintenance machinery (manual_commit_migration.go, orphan cleanup, concurrent-session interleaving) and collapse per-step capture to a thin transcript/prompts accumulator. Also handle explain.go's best-effort ListTemporary. — PR: TBD

Guidance / sequencing

• Don't extend #1051's settings further until the runtime seam exists. Land Phase 0 first.
• Don't block the abstraction on rewind removal. Working-tree rewind rides the non-pluggable temporary half and logs-only rewind rides committed reads, so under the default git primary both keep working unchanged while Phases 0–3 ship. (See the rewind-capability decision above for non-git primaries.)
• The pre-removal investigation (validate disk fallback) is future work tied to removal, not a prerequisite for the abstraction.
• Pre-implementation: Phase 1 must open with the full *GitStore method inventory (committed + temporary) so the interface split doesn't strand callers on the concrete type.

Start with cmd/entire/cli/checkpoint/checkpoint.go and committed_reader_resolve.go, then inventory the construction sites and accessor uses named across cli and strategy. Trace the Phase 0 Open, Refs, Repository, and temporary-capability requirements before reviewing the later committed-store and sync phases. Done means the final facade shape is established without a second call-site migration, while legacy Git behavior remains unchanged.