digidem / digidem/comapeo-map-server
fix: sender reports map share completed before the receiver has the map
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 4
- Forks
- 1
- Avg merge
- 2h 18m
- Merged PRs (30d)
- 7
Description
Filing this for visibility rather than as something to act on. In realistic use maps are 200 MB+ and the practical impact is negligible, so this is probably not worth fixing. It is very visible when testing with small maps, though, and the behaviour is surprising enough that it is worth having written down.
What happens
A sender can report a share as completed before the receiver has the map, and even when the receiver never gets it at all. The sender's completed is set in DownloadResponse's transform flush() (src/lib/map-share.ts:152), which fires once the map file has been piped into the transform — that is, handed to the HTTP layer. It says nothing about whether those bytes reached the socket, the wire, or the receiver. If the receiver aborts after the sender has flushed, the sender still reports completed while the receiver reports aborted and has written nothing.
The window is bounded by the buffering in the chain — the transform queue, node's outgoing buffer, the kernel send and receive buffers — so a few MB at most, and it does not grow with the file. On a 200 MB map completed fires once essentially everything has landed. On a small map it can fire immediately, which is how this surfaced: raising the read buffer sizes in #70 made the sender fast enough to flush a 1.6 MB test fixture before an abort request could round-trip, and two cancellation tests started failing on Linux.
There is a second, related gap. If the receiver gets every byte but the archive fails validation, the sender reports completed while the receiver reports error.
Why there is no cheap local fix
Waiting on the node response's finish event or on socket drain only moves the line from node's buffer to the kernel's buffer, which still is not receipt — and DownloadResponse has no handle on the node response anyway, since the whatwg-node adapter owns the pump. Watching for the connection to close does not work either: the receiver's undici returns the socket to its pool after reading the body, so no close event arrives.
Proposed fix, if it is ever worth doing
Only the receiver knows the transfer succeeded, and its notion of success is later than mere receipt: pipeTo resolves after the sink's close() (src/context.ts:172), which validates the archive and renames the temp file over the live map. That is the moment worth reporting back.
Add POST /mapShares/:shareId/complete alongside the existing /decline, behind the same validateRemoteDeviceId guard. The receiver calls it from DownloadRequest#start where it currently sets its own completed (src/lib/download-request.ts:158), reusing the anyFetch and SecretStreamAgent plumbing it already uses to POST /decline. The sender transitions on the ack rather than on flush. Carrying the outcome in the ack rather than a bare "done" closes the validation-failure gap too.
Peers run different app versions, so the receiver should opt in on the /download request via a header or query flag. A sender that sees the flag waits for the ack; a sender that does not behaves exactly as it does today. That keeps it compatible in both directions with no flag day.
Two decisions already made: the sender stays in downloading between flush and ack, sitting at 100% for a moment, rather than gaining a sent state; and if the ack never arrives — receiver crashed or lost the network after receiving everything — the share goes to a new unknown state rather than falling back to an optimistic completed. That adds one status to the public MapShareState union, which is an API change for consumers.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/lib/map-share.ts:152, src/context.ts:172, and src/lib/download-request.ts:158, then inspect the existing /decline request, validateRemoteDeviceId guard, anyFetch, and SecretStreamAgent plumbing. Done means sender completion follows receiver validation, failures are carried in the acknowledgement, older peers remain compatible, and the new unknown state is represented in the public MapShareState union.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- api, backend, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100