Clarification on WARN for lagging partitions/groups
- Dominant language
- Go
- Stars
- 4k
- Forks
- 818
- Avg merge
- 1h 14m
- Merged PRs (30d)
- 1
Description
Hello,
I'm running a POC for Burrow and can't seem to get a WARN status to appear for my consumer group or any partitions. I've extensively read the [Evaluation Rules](https://github.com/linkedin/Burrow/wiki/Consumer-Lag-Evaluation-Rules), and here's my understanding:
By default, Burrow stores 10 offsets per partition, then using this and the consumer commit interval, determines the size of the evaluation window. I also understand that between _every_ committed offset, the lag needs to either stay the same or increase in order for the group/partition to be considered in a warning state.
I've setup a consumer that is purposely slow, and have a producer that is producing considerably faster. Despite this, Burrow always reports the status to be OK. Even with a lag of 200000+
I keep running over things in my head and trying to tweak my application, but nothing seems to work. I'm hoping someone would be willing to help shed some light on the issue, maybe I'm missing something. Thanks!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with Burrow's Evaluation Rules and reproduce the report using the described slow consumer and faster producer. Determine why a large lag still reports OK, then document the expected WARN behavior or identify the discrepancy that needs fixing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kafka
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100