Peekaping is stuck once a monitor goes down (for whatever Reason)
- 主要言語
- Go
- スター
- 1.2k
- フォーク
- 69
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
**Describe the bug**
Peekaping is stuck once a monitor goes down (for whatever Reason). I'm not entirely sure why, but once a monitor (Ping or HTTP) goes down, it doesn't come back up (almost ever).
I have also `gatus` and `uptime-kuma` monitoring those Hosts so I know for a Fact that most of them are 100% up.
Regardless of why the monitor came down, it should continuously be pinged (ICMP Ping) / queried (HTTP). Instead, after a while, **Peekaping just gives up even trying anymore**, thus the Monitor is and stays down forever.
**Database**
- postgresql
**Version**
- Peekaping: 0.0.45
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
Monitor should continuously be pinged (ICMP Ping) / queried (HTTP) even after it goes down.
This is what `gatus` and `uptime-kuma` do out of the Box.
Peekaping instead seems to get stuck and/or give up after a while 😕.
**Screenshots**
Time is currently between 08h06 (when I started writing this Issue) and 08h22 (when I finished writing this Issue).
Ping Monitor: Peekaping has been stuck / locked for the past 1+ Hour:
```
podman@peekaping:~/containers/quadlets/peekaping$ journalctl --user -xeu peekaping-ingester --no-pager | grep -i "jk-bms-bat01"
Mar 08 06:09:22 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946562.0153909,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.011741054}
Mar 08 06:09:41 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946581.5432782,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.003792322}
Mar 08 06:10:01 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946601.3335428,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.029473007}
Mar 08 06:10:21 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946621.4557583,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.019578605}
Mar 08 06:10:40 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946640.2602067,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.013078612}
Mar 08 06:11:01 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946661.2837162,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.02184739}
Mar 08 06:11:21 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946681.1704283,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.013652279}
Mar 08 06:11:40 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946700.7179687,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.019816899}
Mar 08 06:12:01 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946721.4598243,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.024424851}
Mar 08 06:12:21 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946741.967585,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.006606443}
Mar 08 06:12:40 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946760.8822973,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.015436036}
Mar 08 06:13:00 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946780.4174168,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.017082199}
Mar 08 06:13:20 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946800.9011767,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.013009503}
Mar 08 06:13:41 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946821.9405189,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.020048074}
Mar 08 06:14:00 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946840.8907442,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.018757894}
Mar 08 06:14:21 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946861.033639,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.012397571}
Mar 08 06:14:41 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946881.866438,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.013919811}
Mar 08 06:15:01 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946901.4896088,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.002308295}
Mar 08 06:15:21 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946921.9233463,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.012852423}
Mar 08 06:15:41 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946941.1913452,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.038515999}
Mar 08 06:16:01 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946961.8891952,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.00965311}
Mar 08 06:16:20 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772946980.8768117,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.018421893}
Mar 08 06:16:41 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772947001.5268166,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.006403313}
Mar 08 06:17:02 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772947022.151422,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.027069306}
Mar 08 06:17:22 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772947042.1583946,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.016098031}
Mar 08 06:17:41 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772947061.6278179,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.004193929}
Mar 08 06:18:01 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772947081.3016024,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.019444969}
Mar 08 06:18:38 peekaping peekaping-ingester[125362]: {"level":"info","ts":1772947118.0940034,"caller":"ingester/handler.go:96","msg":"Successfully processed ingester task","svc":"peekaping:ingester","component":"ingester_handler","monitor_id":"139b03a6-1a34-42e8-86b4-74a989f1b6e9","monitor_name":"jk-bms-bat01","duration":0.011185254}
podman@peekaping:~/containers/quadlets/peekaping$ date
Sun Mar 8 08:15:57 AM CET 2026
```
HTTP Monitor:Peekaping has been stuck / locked for the past 7+ Hour:
For some weird Reason it says the Certificate Expiry Notification is enabled even though it clearly is not:
```
podman@peekaping:~/containers/quadlets/peekaping$ journalctl --user -xeu peekaping-producer --no-pager | grep -i "tang1"
Mar 08 08:07:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953620.0261664,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:07:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953650.026036,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:08:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953680.0305603,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:08:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953710.0372384,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:09:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953740.010066,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:09:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953770.0203052,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:10:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953800.0368657,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:10:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953830.043507,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:11:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953860.054222,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:11:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953890.0141444,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:12:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953920.0385141,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:12:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953950.041805,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:13:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772953980.0254714,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:13:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954010.0441544,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:14:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954040.052076,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:14:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954070.0187242,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:15:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954100.0201797,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:15:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954130.014118,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:16:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954160.0591426,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:16:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954190.0177045,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:17:00 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954220.0287147,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
Mar 08 08:17:30 peekaping peekaping-producer[125344]: {"level":"debug","ts":1772954250.0288277,"caller":"producer/processor.go:197","msg":"Certificate expiry checking configured","svc":"peekaping:producer","component":"producer","monitor_id":"f36d7225-4e81-4a0f-8d66-f154b347c477","monitor_name":"tang1","check_cert_expiry":false}
podman@peekaping:~/containers/quadlets/peekaping$ date
Sun Mar 8 08:17:54 AM CET 2026
```
**Desktop (please complete the following information):**
- OS: Ubuntu GNU/Linux 24.04 AMD64
- Browser: Brave 1.87.192 (Official version) (64-bit), based on Chromium 145.0.7632.160
**Smartphone (please complete the following information):**
N/A (only using Desktop for now)
**Additional context**
Add any other context about the problem here.
EDIT 1: from inside the Containers / Pod, after installing `curl`, I can `curl` the HTTP Endpoints without Issue. `ping` works fine. DNS Resolution works without Issues.
So ... it's something wrong with Peekaping's Binary 😕.
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
調査の方向性
The issue points to the ingester component (ingester/handler.go) and producer (producer/processor.go). Start by examining the task scheduling logic in the ingester to see why it stops processing after a monitor goes down. Look for any state that might cause tasks to be dropped or paused. Check the database for monitor status flags. Reproduce by setting up a test monitor and simulating a failure.
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- go, postgresql
- 領域
- backend, observability
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 35/100