GoogleCloudPlatform / GoogleCloudPlatform/esp-v2
endpoints-runtime envoy startup frequently nonresponsive or segfaults on GKE
- Dominant language
- Go
- Stars
- 307
- Forks
- 185
- Avg merge
- 14h 45m
- Merged PRs (30d)
- 6
Description
Running [endpoints-runtime:2.52](https://console.cloud.google.com/artifacts/docker/endpoints-release/us/gcr.io/endpoints-runtime/sha256:d2082668e5f7e33492041458867fa46bf5f7c0bdc6ccd8e58254486470ac5bd6?inv=1&invt=Ab1p4Q) in sidecar mode on Autopilot GKE fails to reach a healthy state with increasing frequency over the past year or so.
Once the sidecar reaches a healthy state, it tends to stay healthy, even for weeks at a time. However, the transient startup issues are frequent enough now that we don't want to risk changing a deployment because the ESPv2 sidecar cannot be trusted to reach a healthy state.
More and more often, our deployment logs segmentation faults from the Envoy process, as seen in the image.
The instances of segmentation faults do not have consistent backtraces across instances.
I know very little about Envoy, but FWIW, https://github.com/envoyproxy/envoy/issues/27775 expresses similar symptoms with a suggested root cause.
Is this a known issue? Any help would be appreciated.
Contributor guide
Assessment
This issue has not been assessed yet.