influxdata / influxdata/kapacitor

Twice a reset (OK) in TICK script fails when the alarm is raised in another script

Open
#2,616 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
2.4k
Forks
479
Avg merge
4d 16h
Merged PRs (30d)
4

Description

Hello,

We have 2 scripts, script A that triggers an Alarm and script B that clears (OK) the Alarm using the unique alarmId. We observe that only the first time the Alarm is cleared by script B, this works fine but any subsequent time to clear it fails. We have multiple scripts that have this common behaviour...
So:
at time 1: Script A: trigger Alarm
at time 2: Script B: clear Alarm (this works)
at time 3: Script A: trigger Alarm (this works)
at time 4: Script B clear Alarm (fails, the alarm will remain at the previous level)

This doesn't happen when the triggering and clearing is within the same script. Any reason why this occurs, is it a bug?

Sample code for the reset:
`var data=stream
|from()
.measurement('clear_function')
|alert()
.critReset(lambda: TRUE)
.warnReset(lambda: TRUE)
.infoReset(lambda: TRUE)
.id('{{ index .Tags "alarmId" }}')
.message('Manual clear of alarmId {{ index .Tags "alarmId" }} by {{ index .Fields "user"}}')
.topic('prod.em_raw_alarms')`

Best regards,
Menno

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the sequence in the issue with scripts A and B: trigger the alarm, clear it by unique alarmId, trigger it again, and clear it again. Focus on the alert() configuration using critReset, warnReset, infoReset, and id in the provided TICK script. Done means repeated clears from the separate script reset the alarm rather than leaving it at the previous level.

Written by the indexing model from the issue text.

Assessment

Domain
observability-sre
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.