influxdata / influxdata/kapacitor
high CPU usage after SIGHUP reload
- Dominant language
- Go
- Stars
- 2.4k
- Forks
- 479
- Avg merge
- 4d 16h
- Merged PRs (30d)
- 4
Description
Do you have ideas why sometimes after "reload" whole Kapacitor system blocked and CPU usage spike appears?
```
ts=2018-09-27T13:34:42.754Z lvl=info msg="SIGHUP received, reloading tasks/templates/handlers directory..." service=run
ts=2018-09-27T13:34:42.811Z lvl=info msg="stopped task" service=kapacitor task_master=main task=alert_cpu
ts=2018-09-27T13:34:42.851Z lvl=info msg="started task" service=kapacitor task_master=main task=alert_cpu
ts=2018-09-27T13:34:42.912Z lvl=info msg="stopped task" service=kapacitor task_master=main task=alert_disk
ts=2018-09-27T13:34:42.948Z lvl=info msg="started task" service=kapacitor task_master=main task=alert_disk
ts=2018-09-27T13:34:42.998Z lvl=info msg="stopped task" service=kapacitor task_master=main task=alert_disk_forecast
ts=2018-09-27T13:34:43.027Z lvl=info msg="started task" service=kapacitor task_master=main task=alert_disk_forecast
ts=2018-09-27T13:34:43.169Z lvl=info msg="stopped task" service=kapacitor task_master=main task=alert_influxdb_series
ts=2018-09-27T13:34:43.197Z lvl=info msg="started task" service=kapacitor task_master=main task=alert_influxdb_series
ts=2018-09-27T13:34:43.253Z lvl=info msg="stopped task" service=kapacitor task_master=main task=alert_iowait
ts=2018-09-27T13:34:43.289Z lvl=info msg="started task" service=kapacitor task_master=main task=alert_iowait
```

seems whole system blocked for several minutes: it leads to False-positive deadman alerts trigger :(
note: if Reload kapacitor one more time after several minutes - everything is OK (cpu usage in normal state)
Contributor guide
Research direction
The report only identifies SIGHUP reload behavior and includes Kapacitor logs; no source files or tests are named. Start by reproducing a reload while monitoring CPU and task activity, then trace the reload path and compare normal and high-CPU runs. Done means the cause is identified and reload no longer blocks the system or triggers false-positive deadman alerts.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100