Explore MPMC queue contention, wakeup, and storage optimizations
- Dominant language
- Rust
- Stars
- 269
- Forks
- 38
- Avg merge
- 16h 44m
- Merged PRs (30d)
- 102
Description
## Goal
Explore measurable performance improvements to the competing MPMC queues introduced in #265. Preserve their delivery, strict bounded capacity, cancellation, and disconnection contracts. The existing mutex-based implementation is a valid baseline; a lock-free rewrite is not a prerequisite.
## Candidates to investigate
1. **Queue and waiter coordination.** [Shared](https://github.com/apache/asyncband/blob/9319ab49dfe24807840dcdbb6694e7f72cb66873/asyncband/src/mpmc/queue.rs) uses a queue mutex plus separate sender and receiver semaphores, each with its own waiter lock. Profile lock acquisitions, cache contention, and retries after a notified task loses the race. Compare this with keeping queue state and waiter bookkeeping under one mutex before introducing more elaborate synchronization.
2. **Separate bounded and unbounded internals where useful.** Both currently share `capacity: Option` and both semaphore fields, although unbounded sends never wait for capacity. A smaller unbounded state and a bounded-specific capacity model may simplify hot paths. Public endpoint types can stay unchanged.
3. **Bounded storage allocation.** The current `VecDeque` grows lazily while the queue mutex is held. Compare preallocation or a fixed-capacity buffer with the current approach, including construction cost and large, mostly empty capacities. Preserve the exact requested logical capacity.
4. **Unbounded burst handling and memory retention.** Measure allocation frequency and retained memory after a burst drains. Investigate chunked storage, incremental reclamation, or limited internal batching if measurements justify them. MPSC's receiver-private batch relies on a unique receiver; do not copy it into MPMC without preserving message accessibility and correct cancellation/drop behavior for competing receivers.
5. **Waiter allocation and scheduling overhead.** Profile repeated registration, redundant waker clones, and wake-to-poll round trips under empty/full transitions. Optimize only measured costs while preserving notification transfer on cancellation and the Waker contract in `AGENTS.md`.
These are hypotheses to test, not a requirement to implement every candidate. Coordinate capacity-accounting changes with the bounded-reservation work in #297.
## Measurement and acceptance
Start with the [existing MPMC ecosystem benchmarks](https://github.com/apache/asyncband/tree/9319ab49dfe24807840dcdbb6694e7f72cb66873/benchmarks/ecosystem/mpmc), which compare asyncband, async-channel, and Flume across 1P/1C, 1P/8C, 8P/1C, and 8P/8C. Keep producer/consumer behavior and message counts comparable.
- Cover bounded and unbounded queues, small and larger capacities, steady traffic, bursts, and idle-to-active transitions. Include workloads where receivers stop at different times rather than only consuming equal quotas.
- Report reproducible before/after results with commit, toolchain, hardware, runtime/thread setup, and benchmark parameters. Measure throughput and latency; include allocations and peak/retained memory for storage changes.
- Explain which cost each change removes and report regressions across the other topologies. Land independently reviewable improvements rather than one combined backend rewrite.
- Preserve deterministic cancellation and disconnect regressions, exactly-once delivery, and bounded-capacity checks; use the relevant `cargo x` workflows and Miri where appropriate.
Follow-up to #211 and #265; part of #206. This issue does not add public APIs.
Contributor guide
No contributing guide indexed for this repository
Research direction
Read asyncband/src/mpmc/queue.rs and the existing benchmarks under benchmarks/ecosystem/mpmc first. Establish comparable bounded and unbounded measurements across the listed producer/consumer topologies, then use the relevant cargo x workflows and Miri for cancellation, disconnect, delivery, capacity, and Waker regressions. Done means a focused, independently reviewable improvement with reproducible throughput, latency, and allocation or memory results, without changing public APIs or queue contracts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- distributed-systems, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100