Azure / Azure/azure-verified-modules-tools
Watch for AVM Terraform modules with unreleased fixes
- Dominant language
- PowerShell
- Stars
- 2
- Forks
- 5
- Avg merge
- 12h 51m
- Merged PRs (30d)
- 30
Description
## Summary
Nothing watches for AVM Terraform modules whose fixes are merged but never released. The result is issues closed as ✅ fixed where the fix is not installable from the registry — in the two confirmed cases, for nearly a year.
This proposes a scheduled, deterministic workflow (plain `bash` + `gh` + `jq`, **no LLM**) that reports the gap.
## Why this exists
AVM's [Terraform issue-triage guide](https://raw.githubusercontent.com/Azure/Azure-Verified-Modules/main/docs/content/help-support/issue-triage/terraform-issue-triage.md) says:
> Only close the issue, once the next version of the module was fully developed, tested and published.
GitHub's default behaviour contradicts that: merging a PR whose body says `Fixes #N` closes the issue immediately, whether or not a release follows. Following the rule takes deliberate effort, and AVM's docs never connect the rule to the label that represents the state — `Status: Awaiting Release To Be Cut :scissors:`, defined in [`avm-standard-github-labels.csv`](https://raw.githubusercontent.com/Azure/Azure-Verified-Modules/main/docs/static/governance/avm-standard-github-labels.csv) as *"This is fixed in the main branch but not in the latest release, will be fixed with next release cut"*.
In practice the gap is usually small. Across 30 merged PRs carrying closing keywords in 10 repos, the median merge→release gap was **0 days** (73% within 24h; `portal-dashboard` PR #103 merged 17:44, `v0.2.0` published 17:56). There is no release automation in these repos — maintainers simply treat merge and release as one action.
But the tail is where users get hurt.
## Measured today (188 non-archived `terraform-azurerm-avm-*` repos)
| finding | count | notes |
| --- | --- | --- |
| **Repos with no releases at all** | **49** | not consumable from the registry at all |
| Repos with commits after their newest release | 138 of 139 | mostly `chore: run avm pre-commit [skip ci]` — not actionable |
| **Closed issues whose fix is in no release** | **2 confirmed** | found in the 30 worst repos only; full sweep not yet run |
The two confirmed cases, each verified by hand:
- `terraform-azurerm-avm-res-desktopvirtualization-workspace#101` — closed 356 days ago, fixing PR #102 still unreleased
- `terraform-azurerm-avm-ptn-avd-lza-managementplane#108` — closed 328 days ago, fixing PR #109 still unreleased
Both read as "closed ✅" to their reporters.
## Proposed check
Per repo, three API calls and zero judgment:
```
newest published release (sort releases by published_at)
-> compare ...HEAD
-> drop chore / pre-commit / deps / [skip ci] commits
-> extract PR numbers from remaining commit subjects
-> for each PR, read closing-keyword refs from its body
-> report the referenced issues that are CLOSED
```
Report three tiers, highest first:
1. **Ghost-closed** — closed issue, fix in no release. Highest value: a user has been told their bug is fixed and it is not shipped.
2. **Ageing unreleased work** — functional commits unreleased past a threshold (suggest 90 days).
3. **Never released** — the 49 repos above. Different problem, same sweep, worth surfacing once.
## Implementation notes
- **Output one rollup issue, updated in place.** Per-repo issues would be ~88 notifications nobody reads.
- **Reuse `repository-management/repository-sync/actions/avm-repos`** for enumeration, archived-repo skipping and app auth. Same pattern as `repository-management-sync.yml`.
- **Sort releases by `published_at`; do not use `/releases/latest`.** It disagrees on at least one repo: `terraform-azurerm-avm-res-netapp-netappaccount` answers `0.2.0` (2025-05-09) while `v0.3.0` (2025-12-02) exists, both non-draft and non-prerelease. Comparing against a stale tag makes released fixes look unreleased. (Same fix applied to the triage workflow in Azure/azure-verified-modules-managed-files#42.)
- **Match on PR number, not merge SHA.** Squash merges rewrite the SHA — on `netapp-netappaccount`, PR #31's `merge_commit_sha` is not the commit on `main`; the work landed under a different PR number entirely. Commit-subject PR numbers survived that; SHAs did not.
- **Filter the literal `#123`.** AVM's PR template contains `Closes #123` as placeholder text, which produces false positives on every repo.
## Why not an agent
The computation has an exact answer and needs no reasoning. An LLM would add ~30 AIC per repo (~$100/sweep at 188 repos) plus a hallucination surface to what is a `jq` filter. The only genuinely judgment-based question — whether unreleased work is *worth* cutting a release for — belongs to the module owner, not a bot.
## Caveat on scope
The "functional vs chore" classification above is approximate; an initial filter counted `chore: pre-commit updates` and `fix: grept apply` as functional. The chore patterns need tightening before the ageing tier is trustworthy. The ghost-closed tier does not depend on that classification and is solid.
Ghost-closed numbers come from the 30 worst repos only — a full sweep may find more.
Contributor guide
Research direction
Start with repository-management-sync.yml and the repository-management/repository-sync/actions/avm-repos action to understand scheduled enumeration, authentication, and archived-repository handling. Implement the deterministic release-to-HEAD and closed-issue check, then verify that one rollup issue is updated with ghost-closed, ageing unreleased, and never-released findings while excluding placeholder PR references and chore commits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, github-actions, shell
- Domain
- ci-cd, devops, release
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 65/100