chart-version-bump and stack-pin-bump silently drop queued runs on simultaneous releases
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 218
- Forks
- 72
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 427
Description
What happened
When several service or chart releases land within the same window (for
example a combined-PR merge that tags multiple services at once), the
`chart-version-bump.yml` and `stack-pin-bump.yml` workflows can silently
drop some of the bumps they were supposed to apply.
Both workflows share a single, repo-wide, un-scoped concurrency group
(`chart-version-bump` / `stack-pin-bump`). GitHub Actions keeps at most one
queued run per concurrency group: when a new run is queued while another is
already queued (not yet started), the older queued run is cancelled before
it ever runs, with no error surfaced anywhere.
Real occurrence
On 2026-09-04, six services released within about two minutes as part of a
combined merge. Two `chart-version-bump.yml` runs were cancelled while
queued:
- `src/control-plane-services/cloud-functions/v1.17.0`
- `src/control-plane-services/encrypted-secret-store/v0.5.0`
Neither bump landed. `encrypted-secret-store`'s `Chart.yaml` `appVersion`
stayed stale until a manual `workflow_dispatch` re-run recovered it six days
later. Nothing in the pipeline surfaced the gap; both runs showed as
"cancelled" with zero jobs, easy to miss in the Actions history.
Confirmation
The cancelled runs state the cause themselves:
Canceling since a higher priority waiting request for chart-version-bump exists
GitHub names the concurrency group in the message, so this is the group evicting
a queued run rather than anything in the workflow body failing. Both runs record
zero jobs, consistent with cancellation before any job started.
Fix
Scope each workflow's concurrency group per release tag instead of
repo-wide, so simultaneous releases for different services or charts no
longer cancel each other's queued runs. Add a retry loop around the shared
bump branch's push, since per-tag groups mean truly concurrent runs can now
race on that branch; force-with-lease turns a race into a rejected push
that the retry recovers from, rather than a silent drop.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting .github/workflows/chart-version-bump.yml and .github/workflows/stack-pin-bump.yml, then trace the shared bump branch push logic. Scope each workflow's concurrency group by release tag and add retry handling for rejected force-with-lease pushes; done means simultaneous releases no longer cancel unrelated queued runs or leave bumps unapplied.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, github-actions, yaml
- Domain
- ci-cd, devops, release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100