0xMiden / 0xMiden/note-transport-service
StreamNotes resource/DoS hardening (subscriptions, polling, cloning, leaks)
- 主要語言
- Rust
- 星號
- 3
- 分支
- 10
- 平均合併
- 2 小時 23 分鐘
- 30 天內合併 PR
- 4
描述
Severity: high/medium.
### Summary
The streaming subsystem has several independent resource issues; grouping them since they touch the same code and should be fixed together.
### Findings
1. **Unbounded subscriptions.** `stream_notes` registers a sub unconditionally (`crates/node/src/node/grpc/mod.rs:291-311`); no per-connection or global cap. The `GlobalConcurrencyLimitLayer` permit is released once the *response future* resolves (stream established), so it does not bound live streams.
2. **O(distinct-tags) DB polling per tick.** `streaming.rs:96-97` issues one `fetch_notes` per tracked tag, sequentially, every 500 ms, against single-writer SQLite. Subscribing to many distinct tags stretches the tick arbitrarily. The multi-tag `fetch_notes_by_tags` already exists and should be used.
3. **Control-message starvation.** `streaming.rs:220-237` races `query_updates()` (which begins with `sleep(500ms)`, line 93) against `rx.recv()` in a `select!`; every control message (including the per-poll `Waker`, line 287) cancels and restarts the 500 ms timer. Enough control traffic and the DB poll never runs. Use a persistent `tokio::time::interval` outside the select.
4. **Per-subscriber batch cloning.** `streaming.rs:130` does `notes.clone()` per subscriber (up to ~256 MB), so N subscribers on a hot tag multiply memory; each sub's channel buffers up to 32 batches (`mpsc::channel(32)`, `grpc/mod.rs:304`). Share via `Arc`, shrink the buffer.
5. **Subscription leaks / id collisions.** `Sub::drop` uses `try_send` on a capacity-128 control channel (`streaming.rs:297-310`); if full, the RemoveSub is lost and the tag stays polled forever. Subscription ids are `rand::random::()` with no collision check (`grpc/mod.rs:297`), so a collision overwrites another client's entry. Use an atomic counter and a non-lossy control channel.
6. **Full-backlog replay on resubscribe.** When the last sub for a tag drops, `TagData` (and its cursor) is removed (`streaming.rs:175-179`); a new sub recreates it at `cursor: 0` (`streaming.rs:262`), replaying the whole 30-day backlog 500 rows/tick. Cheap amplification loop (connect, drain, disconnect, repeat). Fixed together with per-subscriber cursors (#96).
### Recommendation
Per-connection + global subscription caps; one batched `fetch_notes_by_tags` per tick; a fixed interval decoupled from control handling; `Arc`-shared batches with a small buffer; atomic ids + non-lossy lifecycle channel; per-subscriber cursors.
Related: #96, #28 (client reconnects).
---
Part of #114.
貢獻指南
評估
這個 Issue 還沒有評估資料。