prometheus / prometheus/prometheus

PromQL: add an annotation (warning) for too many resets in a counter

Open
#16,618 4 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
66.1k
Forks
10.8k
Avg merge
2d 1h
Merged PRs (30d)
131

Description

Proposal

Users frequently come with questions about absurd results from rate, etc., when their problem is that the underlying data is oscillating.

Here is an example of oscillating underlying data:
Image

We could warn when this is observed inside PromQL functions that take counters.
For instance, if the query is rate(foo[5m]), and we see 5 data points with 4 resets, that would be "too many".
Some number of resets is expected; a moderate number should not be warned about.

Suppose we set the bar at resets / points > 0.6 ?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The entry point is the PromQL counter functions, with rate(foo[5m]) as the concrete example; begin by tracing how resets and points are counted there. The work is done when excessive reset ratios produce a warning while moderate reset rates do not, with the threshold and affected functions settled.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.