fwcd / fwcd/d2

Add better mechanism for recreating pods on Helm upgrade

Open
#128 0 comments 0 reactions 0 assignees View on GitHub
ci-cd docker
Dominant language
Swift
Stars
17
Forks
5
PR merge metrics
No merged PRs in 30d

Description

We currently use a [timestamp annotation](https://github.com/fwcd/d2/blob/67a25682f80c66c3c5db9af0a464ba18d8d6be1d/Helm/d2/templates/deployment.yaml#L18) in our deployment template (similar to the suggestion in https://stackoverflow.com/a/64151965) to ensure that the `helm upgrade` [run by GitHub Actions](https://github.com/fwcd/d2/blob/67a25682f80c66c3c5db9af0a464ba18d8d6be1d/.github/workflows/deploy.yml#L39) always recreates the pods. However, terminating a D2 pod sometimes takes up to a minute, during which both the old and the new instance may respond to user commands.

Ideally, we could find a way to either wait for the old pod to fully terminate (with the downside of having ~1m downtime per deployment), coordinate termination in a way that the terminating pod no longer responds (perhaps even by using Discord's sharding mechanism?) or restart the pod directly after running `helm upgrade`.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.