google / google/seesaw

Health Check notification - Send failed

Open
#125 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
5.7k
Forks
505
PR merge metrics
No merged PRs in 30d

Description

Consistently see healthcheck notification failed to send in some of the deployments

Client has the following errors in the log

```
E0301 16:41:25.941121 157041 core.go:616] Send failed 1 times: read unix @->/var/run/seesaw/engine/engine.sock?backlog=8192: i/o timeout
..........
E0301 16:45:41.944832 157041 core.go:616] Send failed 9 times: read unix @->/var/run/seesaw/engine/engine.sock?backlog=8192: i/o timeout
E0301 16:46:13.945305 157041 core.go:616] Send failed 10 times: read unix @->/var/run/seesaw/engine/engine.sock?backlog=8192: i/o timeout
```

Server has the /var/run/seesaw/engine/engine.sock connections are piling up

```
ss | grep engine.sock | wc -l
15905
```

Any pointers to why this error happening will be helpful

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.