grafana / grafana/plugin-ci-workflows
CD staleness check can reject newer code when an older release finishes later
- Dominant language
- Go
- Stars
- 10
- Forks
- 6
- Avg merge
- 19h 51m
- Merged PRs (30d)
- 97
Description
We observed an overlapping tagged-release deployment and automatic main-branch deployment leave an environment on the older code.
Let A be the tagged release commit and B a subsequent commit on main. Both deployments target the same plugin and environment:
```text
Time →
t1 t2 t3 t4
Release A: starts ───────────────── updates
env to A
finishes
Main build B: starts ───────────────── checks → skipped
▲
│
B started before A updated the config,
so B is classified as stale.
Source order: A → B (B contains newer code)
Result: environment stays on A.
```
Here, “updates env to A” means updating the environment's version configuration, not the exact time all running instances switch versions.
The staleness check compares the environment configuration file's latest commit timestamp with the deployment workflow's creation timestamp. In this sequence, it treats **“configuration changed after this workflow started”** as evidence that **“this deployment has been superseded.”**
But that configuration change came from the older release, not from a deployment containing B or a later commit.
There is a per-plugin/environment mutex, but preventing concurrent deployment steps does not establish source revision order. The failure above does not require clock skew.
How is deployment ordering intended to work when tagged-release and main-branch workflows overlap? In this case, the newer main build never reached the environment.
Related:
- https://github.com/grafana/plugin-ci-workflows/issues/207
- https://github.com/grafana/plugin-ci-workflows/issues/252
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the deployment workflow's staleness check and the per-plugin/environment mutex described in the issue, then compare their behavior for overlapping tagged-release and main-branch runs. Confirm the failure sequence with commits A and B; done means the newer main build is not classified as stale because an older release updated the environment configuration.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions
- Domain
- ci-cd, devops, release
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100