benbjohnson / benbjohnson/litestream
S3 manifest: non-manifest-aware writers can silently stale an enabled manifest
- Dominant language
- Go
- Stars
- 14.4k
- Forks
- 417
- Avg merge
- 7d 1h
- Merged PRs (30d)
- 21
Description
Follow-up to the S3 manifest feature (PR #1173 / #1172). Does not block that PR.
## Problem
The manifest consistency protocol assumes every writer against a replica is manifest-aware: each mutation acquires the `.manifest/lock.json` lease, writes a version-0 invalidation sentinel, and republishes a fenced manifest.
A writer that does not have the manifest capability enabled skips all of that (`ReplicaClient.WriteLTXFile` / `DeleteLTXFiles` short-circuit when all manifest flags are false) and mutates LTX objects without invalidating an existing `manifest.json`. A manifest-reading replica would then trust a stale manifest that is missing the new file (or still lists a deleted one) until the next manifest-aware mutation.
Concrete path: the `litestream-vfs` write path builds its replica client via `NewReplicaClientFromURL` and passes it to `NewVFS` without enabling any manifest flag, then calls `WriteLTXFile`. If that client writes to a bucket whose manifest is enabled by another (manifest-aware) writer, it can stale the manifest.
## Options
- Document explicitly (done in the PR #1173 review): do not mix a non-manifest-aware writer with a manifest-enabled replica.
- Consider a guard: a manifest-aware client could detect concurrent non-participating writes (e.g. an unexpected object count vs. manifest) and fall back to LIST, or a writer could refuse when it sees a foreign manifest it is not maintaining.
- Longer term, making manifest support available to every client type would remove the asymmetry.
## Severity
Low in practice (mixing a VFS writer with a manifest-enabled S3 replica is an unusual config), but it is a silent-correctness footgun worth tracking.
Contributor guide
Research direction
Start with ReplicaClient.WriteLTXFile, DeleteLTXFiles, NewReplicaClientFromURL, and NewVFS, then read the manifest protocol around .manifest/lock.json. The issue has no selected implementation: determine whether the outcome is a restriction, a guard or fallback, or broader client support. Done means the chosen approach prevents or clearly documents stale manifests when writers are mixed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100