Scheduler consumer registers a new UUID consumer in the Redis group on every poll and never deletes them
- Dominant language
- Rust
- Stars
- 43.7k
- Forks
- 5.1k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 205
Description
The scheduler consumer creates a fresh consumer identity on every poll iteration and registers it in the Redis consumer group. Redis never expires consumer entries, so the group grows without bound for as long as the consumer runs.
## Cause
`crates/scheduler/src/consumer.rs`:
```rust
let consumer_name = format!("consumer_{}", Uuid::new_v4());
```
That is inside the polling loop, and `XREADGROUP` registers the name. Consumer entries are only ever removed by `XGROUP DELCONSUMER`.
There is already a wrapper for that — `consumer_group_delete_consumer` in `crates/redis_interface/src/commands.rs` — and it has no callers anywhere in the tree.
## Measured
On a deployment with the default 5s `loop_interval`, with no tasks flowing at all:
```
XINFO GROUPS :scheduler_stream -> consumers 142
... 20 seconds later ...
XINFO GROUPS :scheduler_stream -> consumers 146
```
One per poll, roughly 17,000 per day, indefinitely. After ~45 minutes of an idle consumer the group held 555 entries.
Nothing fails abruptly. The group metadata just grows, and the bookkeeping behind `XREADGROUP`/`XINFO`/`XAUTOCLAIM` gets progressively heavier — on a Redis that may be shared with other workloads.
## Suggested fixes
Either would do:
- Reuse a stable consumer name per process (for example derived from hostname or generated once at startup) rather than per iteration. This seems closest to the intent of a consumer group.
- Or call the existing `consumer_group_delete_consumer` after a batch completes, for consumers with no pending entries.
If it helps, the safe shape for an external sweep is to delete only consumers where `pending == 0` and idle exceeds some threshold well above the poll interval — deleting a consumer that still holds unacknowledged entries drops those tasks, which on this stream means a payment that silently never retries.
## Versions
Observed on `v1.123.1`; the code is unchanged on `main` at time of writing.
Contributor guide
Research direction
Start in crates/scheduler/src/consumer.rs at the consumer_name creation inside the polling loop, then inspect consumer_group_delete_consumer in crates/redis_interface/src/commands.rs. Compare stable-name reuse with safe deletion of consumers that have no pending entries. Done means the Redis consumer group no longer gains an entry on every poll without risking unacknowledged tasks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- redis, rust
- Domain
- backend, databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100