influxdata / influxdata/kapacitor
alert with stateChangesOnly doesn't trigger OK on script start
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
There are 2 related scenarios here.
1. Lets say you have an alert with `stateChangesOnly` enabled, and kapacitor has triggered a CRITICAL alert. But then you stop the script, or kapacitor, or the whole host it's running on. While the script is stopped the condition clears. When kapacitor starts back up, it doesn't send an OK alert, and so any external systems still see the state as CRITICAL.
2. Similarly you have an alert with `stateChangesOnly` enabled, but you have a bad tick script which results in some erroneous alerts. So you fix the script and then redefine it in kapacitor. When the new script is started, it comes up in OK state, but an OK alert doesn't get triggered. An alert gets triggered if it's still in an INFO, WARNING, or CRITICAL state, just not OK. This results in the false alerts not getting cleared.
I think it would be a good idea to allow triggering an OK alert when a script first starts up.
Contributor guide
Assessment
This issue has not been assessed yet.