Lag summary per topic [question]
- Dominant language
- Go
- Stars
- 4k
- Forks
- 818
- Avg merge
- 1h 14m
- Merged PRs (30d)
- 1
Description
Hello
Currently when querying for lag i receive information in form:
"status" -> "partitions" -> "partition topic".
When topic is located on multiple partitions i need to summarize (correct me if i am wrong) current_lag from all partition containing particular topic.
The issue that when gathering data using collectd topic is part of partition hash:
`{
"topic": "sometopic",
"partition": 1,
"owner": "",
"status": "OK",
"start": {
"offset": 25836605,
"timestamp": 1519389278989,
"lag": 0
},
"end": {
"offset": 25836701799,
"timestamp": 1519389368055,
"lag": 0
},
"current_lag": 190,
"complete": 1
},
`
making it impossible to sum current_lag value from only single topic.
Am i missing a simple way of getting summary lag per topic using ELK + collectd?
Contributor guide
No contributing guide indexed for this repository
Research direction
No source file or test is named. Start by reproducing the partition-level lag response described in the issue and checking how Burrow output is consumed by collectd and ELK. Done should be a confirmed answer about topic-level aggregation, with any required implementation or usage guidance explicitly identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch, go, kafka
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100