redpanda-data / redpanda-data/observability
producer/consumer metrics using redpanda_kafka_request_bytes_total are wrong
Nobody has claimed this yet.
- Dominant language
- Jsonnet
- Stars
- 52
- Forks
- 12
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 1
Description
This metric incorrectly calculates the usage when a learner event is happening (decom, node add, etc). we should be using these instead for determining on-the-wire traffic for the cluster for produce/consume side throughput.
The metric should update to:
Producer traffic:
sum(rate(redpanda_rpc_received_bytes{redpanda_server="kafka", redpanda_id="$redpanda_id"}[5m])) by (cluster)
Consumer traffic:
sum(rate(redpanda_rpc_sent_bytes{redpanda_server="kafka", redpanda_id="$redpanda_id"}[5m])) by (cluster)
Adjust the labels accordingly to fit the observability repo dashboards.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named in the issue. Search the observability dashboard definitions for redpanda_kafka_request_bytes_total, inspect the existing label shape, then compare the rendered panels with the specified received- and sent-byte queries; done means producer and consumer panels show cluster throughput with compatible labels.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- prometheus
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 52/100