Rate-limit Event Ledger writes for repeated status events
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
Description
Streaming sessions can send the same status event every second. Persisting every update creates unnecessary write load.
Add a local cache to each Event Ledger instance with the following behavior:
- On a cache miss, persist the event immediately and cache it.
- On a cache hit, replace the cached value only if the event is newer and mark the entry as pending.
- Persist the latest pending value no more than once every 60 seconds per event key and per service instance.
- Schedule each entry separately, using a timing wheel or similar deadline scheduler, so writes are spread across the 60-second window.
- Use the event timestamp when persisting so an older event cannot overwrite a newer value.
- Remove inactive entries and enforce a maximum cache size.
Acceptance criteria
- The first event for a key seen by an instance is persisted immediately.
- Repeated events for the same key produce at most one additional write every 60 seconds per instance.
- Only the newest pending event is persisted.
- Pending state is cleared only after a successful write.
- Events received during a flush are not lost.
- Out-of-order events cannot replace newer state.
- Cache hits, misses, evictions, flushes, and failures are exposed as metrics.
- Tests cover cache misses, repeated updates, concurrent updates, write failures, expiry, and multiple instances.
The 60-second limit applies per service instance. A global rate limit across all instances is not part of this change.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Event Ledger implementation and its existing tests, then trace the per-instance write path. Define how cache state, separate scheduling, timestamp ordering, eviction, and metrics fit together before implementing. Done means the acceptance criteria pass, including concurrent updates, failures, expiry, and multiple instances.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, distributed-systems, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100