Transport changes cancel all IO loops, retire affected transports cooperatively instead
- Dominant language
- Rust
- Stars
- 929
- Forks
- 143
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 53
Description
Any transport change (synced add or removal, delete_transport(), ...) currently restarts the whole IO scheduler. `Scheduler::stop()`
races every loop future against a cancellation token, so the restart drops IMAP loops at arbitrary await points and reconnects every transport even when only one changed.
#8499 moved the restart trigger to a safer point: the fetch loop that processed the transport sync message defers the restart until its iteration's persistence is committed. That fixed losing the sync message's own remaining processing, but it is not a complete solution: The restart still cancels the loops of the other transports at arbitrary points because only the triggering loop's iteration is protected.
On hard-cancellation all connections are torn down and rebuilt for every transport change and almost all `await` points in the IO loops must reckon with getting arbitrarily cancelled when transport config changes, producing hard to replicate and hard to debug problems, apart from potentially user visible degradations.
I suggest to **reconcile transports** instead of stop-and-restarting-all transports:
- just spawn loops for added transports,
- retire loops of removed or reconfigured transports (via a token that each loop checks at iteration boundaries).
- IMAP fetch loops finish (or fail) on processing their current message and then gracefully exit when the retire-token is present.
- When we need to reconfigure or restart a transport, we can probably retire the current and concurrently start a new IMAP fetch loop already.
- nothing is cancelled at all, unaffected transports keep their connections, and the deferral flag becomes unnecessary.
Note that the SMTP loop singleton needs no real teardown: on a retire-signal it can rebind to the current primary.
Hard cancellation remains only for `stop_io()` and `pause()`, where tearing everything down is the actual intent. Although we could probably also try to stop cooperatively for a few seconds before hard-cancel.
Contributor guide
Research direction
Start by tracing Scheduler::stop() and the fetch loop that processes transport sync messages, then compare their behavior with stop_io() and pause(). Map how transport changes reach the IMAP and SMTP loops. Done means added transports get loops, removed or reconfigured transports retire cooperatively, unaffected transports keep their connections, and hard cancellation remains for stop_io() and pause().
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- networking
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100