influxdata / influxdata/influxdb
Alerts sometimes don't register
- Dominant language
- Rust
- Stars
- 31.7k
- Forks
- 3.7k
- Avg merge
- 13h 37m
- Merged PRs (30d)
- 8
Description
I'm currently using an InfluxDB OSS installation in a Kubernetes cluster (version v2.7.1) with the official helm chart. I've also set up an Edge Data Replication bucket to bring data from my home to my cloud, which is working perfectly so far.
However, I'm now looking to create alerts for when a value hasn't been emitted for a certain amount of time (e.g. 2 hours). The values come from my home-assistant Z-Wave network with the help of zwave-JS-ui. Sometimes, due to radio interference, the devices drop packets and un-join the Z-Wave network, which requires manual resetting. To address this issue, I want to receive a notification in slack to stay up-to-date.
I have set up a deadman check alert to check every minute if a value is emitted for the last 5 minutes (later I will increase the time as said above, when is stable). The problem is that in for some time is working and then somehow it stops registering that a value is critical. I dont have any error in the logs, only in the alert history page that is ok (which isn't). Is there something that am I missing? Is this common behavior.
Contributor guide
Research direction
Start with the deadman check configuration, alert history page, and logs described in the issue, using the Kubernetes deployment and official Helm chart context. Reproduce the transition from working to non-critical behavior and document whether the alert state and Slack notification remain correct when values stop arriving.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- databases, observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100