influxdata / influxdata/kapacitor
Kapacitor Performance Issues And Sluggish Response
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
I am using TICK to monitor my application's performance like time taken for a rest call or time taken by internal method call. I use Kapacitor to trigger an alert when there is an anomaly in time taken by a method call. After round about creating 5 alerts Kapacitor started slowing down and starting closing. When I looked into logs it showed "too many open connections" I sorted this issue by increasing the ulimit but still the sluggishness of Kapcitor remains. Sometimes it sends alerts sometime it does not. Every time I reboot my ec2 instance the alerts get triggered(not all though) but after that alerts dont work. I have valid TICKSCRIPTS and memory utilization also does not cross 1.8%. I am not able to understand problem here. Any help will be much appreciated.
Contributor guide
Research direction
Start by reproducing the slowdown after creating about five alerts, then inspect the logs for the reported "too many open connections" condition and compare behavior before and after the EC2 instance reboot. Done means identifying the cause of the sluggish or missed alerts and confirming that alerts continue to trigger reliably without relying only on a higher ulimit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws
- Domain
- observability
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100