No lag reported in metrics
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 4k
- Forks
- 818
- Avg merge
- 1h 14m
- Merged PRs (30d)
- 1
Description
We are using Burrow, v1.3.4 against Amazon MSK (Apache Kafka), v2.5.1.
Trying to setup burrow/prometheus/grafana monitoring, we faced following issue:
### On stage env
*/metrics* endpoint of burrow shows everything that is expected regarding our custom topics, including:
* **burrow_kafka_consumer_current_offset**
* **burrow_kafka_consumer_partition_lag**
* **burrow_kafka_topic_partition_offset**
REST endpoints show the same, everything looks fine.
### On live env
**/metrics** endpoint shows values of:
* **burrow_kafka_consumer_current_offset** and **burrow_kafka_consumer_partition_lag** for ONLY some internal Amazon MSK topic (__amazon_msk_canary), **BUT NOT FOR ANY CUSTOM TOPIC OF OURS**
* **burrow_kafka_topic_partition_offset** for our custom topics as well as for ___amazon_msk_canary_
What's even more strange, REST endpoints show all as expected for all topics.
After spending a whole day on research (kafka upgrade from v2.2 to v2.5.1, restarting the Burrow, artificially creating the lag by turning off the consumer group) we've totally stuck.
Any help would be appreciated.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Compare Burrow's /metrics and REST endpoint results for custom topics in the reported stage and live environments, using Burrow v1.3.4 with Amazon MSK/Kafka v2.5.1. Start by checking the consumer offset and partition lag metrics against the topic partition offset metrics; done means identifying why live custom-topic consumer metrics are missing and documenting or verifying the resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go, grafana, kafka, prometheus
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100