apache / apache/asyncband

Explore MPMC queue contention, wakeup, and storage optimizations

Open
#298 0 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
Rust
Stars
269
Forks
38
Avg merge
16h 44m
Merged PRs (30d)
102

Description

## Goal

Explore measurable performance improvements to the competing MPMC queues introduced in #265. Preserve their delivery, strict bounded capacity, cancellation, and disconnection contracts. The existing mutex-based implementation is a valid baseline; a lock-free rewrite is not a prerequisite.

## Candidates to investigate

1. **Queue and waiter coordination.** [Shared](https://github.com/apache/asyncband/blob/9319ab49dfe24807840dcdbb6694e7f72cb66873/asyncband/src/mpmc/queue.rs) uses a queue mutex plus separate sender and receiver semaphores, each with its own waiter lock. Profile lock acquisitions, cache contention, and retries after a notified task loses the race. Compare this with keeping queue state and waiter bookkeeping under one mutex before introducing more elaborate synchronization.
2. **Separate bounded and unbounded internals where useful.** Both currently share `capacity: Option` and both semaphore fields, although unbounded sends never wait for capacity. A smaller unbounded state and a bounded-specific capacity model may simplify hot paths. Public endpoint types can stay unchanged.
3. **Bounded storage allocation.** The current `VecDeque` grows lazily while the queue mutex is held. Compare preallocation or a fixed-capacity buffer with the current approach, including construction cost and large, mostly empty capacities. Preserve the exact requested logical capacity.
4. **Unbounded burst handling and memory retention.** Measure allocation frequency and retained memory after a burst drains. Investigate chunked storage, incremental reclamation, or limited internal batching if measurements justify them. MPSC's receiver-private batch relies on a unique receiver; do not copy it into MPMC without preserving message accessibility and correct cancellation/drop behavior for competing receivers.
5. **Waiter allocation and scheduling overhead.** Profile repeated registration, redundant waker clones, and wake-to-poll round trips under empty/full transitions. Optimize only measured costs while preserving notification transfer on cancellation and the Waker contract in `AGENTS.md`.

These are hypotheses to test, not a requirement to implement every candidate. Coordinate capacity-accounting changes with the bounded-reservation work in #297.

## Measurement and acceptance

Start with the [existing MPMC ecosystem benchmarks](https://github.com/apache/asyncband/tree/9319ab49dfe24807840dcdbb6694e7f72cb66873/benchmarks/ecosystem/mpmc), which compare asyncband, async-channel, and Flume across 1P/1C, 1P/8C, 8P/1C, and 8P/8C. Keep producer/consumer behavior and message counts comparable.

- Cover bounded and unbounded queues, small and larger capacities, steady traffic, bursts, and idle-to-active transitions. Include workloads where receivers stop at different times rather than only consuming equal quotas.
- Report reproducible before/after results with commit, toolchain, hardware, runtime/thread setup, and benchmark parameters. Measure throughput and latency; include allocations and peak/retained memory for storage changes.
- Explain which cost each change removes and report regressions across the other topologies. Land independently reviewable improvements rather than one combined backend rewrite.
- Preserve deterministic cancellation and disconnect regressions, exactly-once delivery, and bounded-capacity checks; use the relevant `cargo x` workflows and Miri where appropriate.

Follow-up to #211 and #265; part of #206. This issue does not add public APIs.

Contributor guide

No contributing guide indexed for this repository

Research direction

Read asyncband/src/mpmc/queue.rs and the existing benchmarks under benchmarks/ecosystem/mpmc first. Establish comparable bounded and unbounded measurements across the listed producer/consumer topologies, then use the relevant cargo x workflows and Miri for cancellation, disconnect, delivery, capacity, and Waker regressions. Done means a focused, independently reviewable improvement with reproducible throughput, latency, and allocation or memory results, without changing public APIs or queue contracts.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
distributed-systems, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.