Basekick-Labs / Basekick-Labs/arc
cluster: a manifest UpdateFile (or re-register) for a path whose pull is in flight is deduped away; the reader keeps the old version
- Dominant language
- Go
- Stars
- 677
- Forks
- 53
- Avg merge
- 9h 14m
- Merged PRs (30d)
- 164
Description
Found while reviewing the #759/#795 fix; pre-existing and not changed by it.
`ClusterFSM.applyUpdateFile` (`internal/cluster/raft/fsm.go`) fires the register callback with the comment "content (and checksum) changed, so readers must re-fetch it". The callback is `puller.Enqueue`, and `Puller.enqueue` dedups on the in-flight set: if a pull for that path is already queued or in flight, the update's enqueue returns `SkippedDuplicate` and is dropped.
So a reader whose pull of the old version is in flight when the update commits never re-fetches: the in-flight pull either completes with the old bytes (they still match the old checksum the worker holds) and the reader serves stale content, or fails its checksum against the peer's new bytes and records a failure that only a later pull of the same path clears. Nothing schedules that later pull except the periodic reconciliation pass (default every 5 minutes), and if the reader is inside its startup catch-up the failure counts against the query gate for that interval.
Same shape for a delete-then-re-register of the same path (a rewrite that reuses the name): `ManifestHas` in the #795 fix is path membership only, so it does not distinguish the two versions either.
Fix shape: `enqueue` should compare the incoming entry's LSN/SHA256 with the in-flight request's and, when they differ, mark the in-flight request superseded (the worker discards its result and re-enqueues the new entry), or simply re-enqueue after the current pull finishes. The in-flight map would need to hold the entry, not just the path.
Contributor guide
Research direction
Start in internal/cluster/raft/fsm.go at ClusterFSM.applyUpdateFile, then trace Puller.enqueue and the in-flight request map. Follow the concurrent old-version pull and manifest update paths, including delete-and-re-register, and inspect any existing cluster or puller coverage. Done means a changed LSN/SHA256 is not deduplicated away and the reader does not retain stale content while waiting for reconciliation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100