influxdata / influxdata/kapacitor

Delay in alert generation for deadman kind of task

Open
#2,171 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

Hi
I am seeing intermittent delay in alert generation and I am not able to figure out the reason for it. The delay is about 1 min 30s -2 mins. Its not a big delay, but I would like to understand the reason behind it.

My tickscript is like this-

var template lambda
var orgId lambda
var whereFilter = lambda: template AND orgId

stream
|from()
.where(whereFilter)
.groupBy( 'template', 'orgId')
|stats(30s)
.align()
|derivative('emitted')
.unit(30s)
.nonNegative()
|stateCount(lambda: "emitted" == 0)
|alert()
.stateChangesOnly()
.crit(lambda: "state_count" > 10)
.log('alerts.text')

Contributor guide

Open the contributing guide

Research direction

No files or tests are named. Start by reproducing the supplied tickscript and measuring the timing through stats, derivative, stateCount, and alert; done means identifying and documenting the cause of the reported 1m30s–2m delay.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
observability
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.