MapColonies / MapColonies/infra-tools

Verify Helm values image references against container registries

Open
#17 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ready-for-agent
Dominant language
TypeScript
Stars
0
Forks
0
Avg merge
22h 52m
Merged PRs (30d)
13

Description

## Problem Statement

When editing a Helm values file, there is no way to know whether the image a chart points at actually exists until something tries to pull it — typically a deployment, minutes or hours later. A typo in a tag, a tag that was never pushed, or a repository name that drifted after a rename all look identical to correct values in the editor: plain text with no feedback.

The problem is worse than a plain typo check, because the reference in the file is frequently incomplete on purpose. A chart may carry `repository: my-service` with no host, expecting a registry supplied at deploy time. A tag may be absent entirely, deferring to the chart's `appVersion`. And while developing a chart, the registry named in the file is often not the registry the developer is actually pushing to — the real one only exists at deploy time. So the developer cannot answer "does this image exist?" by reading the file, and neither can any tool that reads only the file.

Separately, developers already hold the credentials needed to answer the question: they are logged in to the relevant registries via `docker login`. That authority sits unused on disk while the editor stays silent.

## Solution

The VS Code extension checks every image reference it finds in a Helm values file against a container registry, using the developer's existing local Docker credentials, and reports the result inline.

Images that exist get a checkmark. Images that do not get a red squiggle naming what is missing — the repository or the tag. Anything the extension cannot determine, whether from an expired token, a missing credential, or an unreachable network, produces no error marker at all, because falsely reporting a good image as missing would destroy trust in the feature faster than any other failure.

To handle the develop-against-a-different-registry case, the workspace can declare a set of registries. When declared, that set replaces whatever registry the values file names, and an image is considered to exist if any registry in the set has it. The UI makes this substitution visible wherever it changes the answer, so a checkmark never quietly means "exists somewhere other than where this file says".

Tagless images are resolved through the governing chart's `appVersion`, matching Helm's own semantics, and the resulting messages say where the tag came from.

The logic lives in two workspace packages so it is reusable outside the editor; only the extension ships in this spec.

## User Stories

### Seeing results

1. As a Helm chart developer, I want a checkmark next to each image reference that exists in a registry, so that I get positive confirmation my values are correct rather than only hearing about failures.
2. As a Helm chart developer, I want a red squiggle on an image reference whose tag does not exist, so that I catch the typo while editing instead of at deploy time.
3. As a Helm chart developer, I want the error message to distinguish "the repository does not exist" from "the tag does not exist", so that I know whether to fix the name or the version.
4. As a Helm chart developer, I want no error marker at all when the extension could not reach or authenticate against a registry, so that I never chase a phantom bug caused by my own expired token.
5. As a Helm chart developer, I want to hover an image reference and see which registry was actually queried, so that I can confirm the check asked the question I thought it asked.
6. As a Helm chart developer, I want to hover an unverifiable image and read why it could not be verified, so that I know whether to log in, check my network, or ignore it.
7. As a Helm chart developer, I want results to appear as I type rather than only on save, so that I learn about a bad tag before committing to it.
8. As a Helm chart developer, I want image errors to appear in the Problems panel, so that they participate in the same workflow as every other diagnostic in my editor.

### Finding image references

9. As a Helm chart developer, I want image references detected wherever they appear in a values file, so that I do not have to place them at conventional paths for the feature to work.
10. As a Helm chart developer, I want subchart values files under a chart directory to be checked too, so that dependency images get the same scrutiny as the parent chart's.
11. As a Helm chart developer, I want files under a chart's `templates` directory to be skipped, so that Go template syntax does not generate meaningless results.
12. As a Helm chart developer, I want unrelated YAML in my workspace left alone, so that opening a CI config or a Kubernetes manifest does not trigger registry traffic.
13. As a Helm chart developer, I want a `repository` key that is not an image reference — a source-control URL, for example — to be left alone, so that the feature does not invent errors in unrelated configuration.
14. As a Helm chart developer, I want a tag written as `1.10` to be checked as `1.10` and not as the number `1.1`, so that YAML's numeric coercion does not cause a false error on a correct file.
15. As a Helm chart developer, I want image references whose values contain Helm template syntax to be skipped silently, so that unresolvable placeholders produce no noise.

### Resolving the registry

16. As a Helm chart developer, I want a repository that already names a host to be checked against that host, so that fully qualified references work without configuration.
17. As a Helm chart developer, I want a registry declared elsewhere in the same values document to be applied to bare repository names, so that charts following the common global-registry convention work as written.
18. As a Helm chart developer, I want to declare a set of registries for my workspace, so that I can verify images while the registry named in the file is not one I can reach.
19. As a Helm chart developer, I want the declared registries to replace the file's registry rather than be checked in addition to it, so that an unreachable production registry does not produce errors while I develop against a development one.
20. As a Helm chart developer, I want an image to count as existing if any of my declared registries has it, so that I do not have to know in advance which one I pushed to.
21. As a Helm chart developer, I want to see which registry a checkmark came from when it differs from the one the file names, so that a passing check never misleads me about where the image lives.
22. As a Helm chart developer, I want the error message to say when a declared registry overrode the file's, so that a squiggle on a line naming production does not make me think production was checked.
23. As a Helm chart developer, I want a status indicator showing that overrides are in effect, so that I can explain why a file is green when I know the image was never pushed to the registry it names.
24. As a Helm chart developer, I want a bare public image name like `nginx` to resolve against Docker Hub, so that common public images are verified without setup.
25. As a Helm chart developer, I want no error when a bare repository name with no configured registry is absent from Docker Hub, so that an internal image the tool had to guess about never gets falsely flagged.
26. As a Helm chart developer, I want to choose between user-level and workspace-level registry settings myself, so that a personal development registry and a team default can coexist.

### Tags from the chart

27. As a Helm chart developer, I want an image with no tag to be checked against the governing chart's `appVersion`, so that the tool follows the same convention my chart does.
28. As a Helm chart developer, I want a values file inside a subchart to resolve `appVersion` from that subchart's own chart metadata, so that the answer matches what Helm would actually deploy.
29. As a Helm chart developer, I want messages about a tag taken from `appVersion` to say so, so that I look in the chart metadata rather than hunting for a tag that is not in the values file.
30. As a Helm chart developer, I want results refreshed when I change a chart's `appVersion`, so that bumping a version does not leave stale checkmarks behind.
31. As a Helm chart developer, I want no marker when there is no chart metadata or no `appVersion` to fall back to, so that the tool stays silent rather than guessing.

### Credentials

32. As a Helm chart developer, I want the extension to use the credentials I already have from `docker login`, so that the feature works with no setup on my part.
33. As a Helm chart developer, I want registries whose credentials live in a credential helper to work, so that the feature is not limited to plaintext credential storage.
34. As a Helm chart developer using Azure Container Registry, I want my identity token exchanged correctly, so that my primary registry is not the one registry the feature fails on.
35. As a Helm chart developer, I want to be told when a registry needs a login, so that I understand why images are unverified rather than assuming the feature is broken.
36. As a Helm chart developer, I want that notification to offer to run the login command for me, so that I can fix it without looking up the registry hostname.
37. As a Helm chart developer, I want to dismiss a registry's login prompt permanently, so that a registry I do not care about stops asking.
38. As a Helm chart developer, I want to be told at most once per registry per session, so that opening several values files does not bury me in notifications.
39. As a Helm chart developer, I want public images checked without credentials, so that public charts work before I have logged in anywhere.

### Overview and navigation

40. As a Helm chart developer, I want a sidebar view listing the image references in the file I am editing with their status, so that I can survey a file's images without scanning it line by line.
41. As a Helm chart developer, I want to switch that view to cover the whole workspace, so that I can find a broken image in a file I do not have open.
42. As a Helm chart developer working in a repository with many charts, I want the view scoped to the current file by default, so that a large repository does not produce an unusable list.
43. As a Helm chart developer, I want expanding a file in the view to be what triggers its checks, so that switching to workspace scope does not fire hundreds of requests at once.
44. As a Helm chart developer, I want clicking an entry to jump to the line in the file, so that I can act on a result immediately.
45. As a Helm chart developer, I want to re-check a single image, a single file, or everything, so that I can confirm a fix without waiting for a cache to expire.

### Practice warnings

46. As a Helm chart developer, I want a warning when a values file pins an image to `latest`, so that the practice is visible during review rather than discovered in production.
47. As a Helm chart developer, I want that warning at a lower severity than a broken reference, so that a style opinion is not confused with a file that will fail to deploy.
48. As a team that deliberately uses `latest` in development values, I want to turn that warning off, so that the feature stays useful without permanent yellow.

### Cost and performance

49. As a Helm chart developer, I want confirmed results cached for a long time, so that reopening files does not repeatedly spend my registry rate limit.
50. As a Helm chart developer, I want negative results cached only briefly, so that an image I just pushed is confirmed shortly after rather than staying red all day.
51. As a Helm chart developer, I want cached results to survive reloading the window, so that restarting the editor does not re-spend my rate limit.
52. As a Helm chart developer, I want registry requests bounded in number, so that opening a values file with many images does not saturate my connection.

### Reuse

53. As a platform engineer, I want the registry-checking logic in a package independent of Helm and of VS Code, so that it can back a CLI or CI check later without being rewritten.
54. As a platform engineer, I want the Helm-specific knowledge in its own package, so that a future CLI does not have to reimplement which chart governs which values file.

## Implementation Decisions

### Packages

Two new workspace packages under `packages/*`, both scoped to the organisation and both private, matching the repo's existing convention that nothing publishes. Both are CommonJS, as ADR 0002 requires of everything in `packages/*` — this is a hard constraint, not a default, because the VS Code extension host cannot load ESM.

- **The Helm package** owns everything that knows what a Helm chart is: locating image references in values file source, and determining which chart's metadata governs a given values file. It never imports `vscode` and never performs I/O directly.
- **The OCI registry package** owns everything that knows what a container registry is: reference normalization, credential resolution, and manifest existence checks. It knows nothing about Helm, YAML, or editors.

The extension composes the two and owns all UI. No CLI work in this spec.

### Finding image references

Detection is structural rather than path-based, so charts do not have to follow a convention to be supported. A candidate is a YAML mapping containing a `repository` key. Because `repository` alone over-matches — a source-control URL under a `repository` key would qualify — a candidate must also carry a corroborating signal: a sibling `tag`, `pullPolicy`, or `registry` key, or a parent key of `image` or one ending in `Image`.

A `tag` sibling is optional, not required, because tagless images resolve through `appVersion`.

Tag and `appVersion` values are read from the **raw source text of the scalar node**, never from the parsed value. YAML coerces `1.10` to the float `1.1` and `12` to an integer; checking the coerced value would produce a confident error on a correct file. This forces a YAML parser that exposes source ranges for scalar nodes, which is required anyway to place diagnostics and decorations.

Values whose text contains Helm template syntax are skipped and produce no marker.

File scope is values files by name, plus any YAML file beneath a directory containing chart metadata, excluding that chart's `templates` directory.

### Chart context resolution

Resolving a tagless image requires the chart's `appVersion`. The governing chart is the one in the **nearest ancestor directory containing chart metadata**, so a values file inside a subchart resolves against the subchart's own metadata rather than the parent's — matching Helm.

This lives in the Helm package, not the extension, with the filesystem read injected as a dependency. The alternative — passing `appVersion` in from the extension — was considered and rejected: "which chart governs this values file" is Helm knowledge, and leaving it in an editor extension guarantees a future CLI reimplements it.

Absent chart metadata or an absent `appVersion` yields no marker.

### Registry resolution

Reference normalization lives in the OCI registry package, not the Helm package, because it is OCI naming semantics rather than anything Helm-specific. The Helm package emits raw strings; the registry package interprets them.

Resolution order when the workspace declares no registries:

1. An explicit host in the repository string — detected by the standard rule that the first path segment contains a dot or a colon, or is exactly `localhost`.
2. A registry declared elsewhere in the same YAML document.
3. Docker Hub.

Single-segment names on Docker Hub receive the `library/` namespace prefix.

When the workspace **does** declare registries, that set **replaces** the file's registry entirely and the reference is checked against every entry. The image exists if any registry returns a match, and the matching registry is reported back so the UI can name it. Replacement rather than union is deliberate: a union would keep firing errors from the unreachable production registry in exactly the development scenario the feature exists to support.

A registry reached only through the Docker Hub fallback — meaning nothing in the file or the settings named a registry — is treated as a guess. A not-found verdict from a guessed registry is **downgraded to unverifiable** and never produces a diagnostic, while a positive match still renders normally. This makes it structurally impossible to combine "we guessed the registry" with "we are confident the image is missing".

Digest-pinned references are out of scope and produce no marker.

### Credentials

The full local Docker credential chain is implemented, because "use local Docker credentials" has a shallow reading that silently fails on real registries:

- Plaintext auth entries, decoded and sent as basic credentials.
- A global credential store, invoked as a `docker-credential-*` subprocess.
- Per-registry credential helpers, same mechanism.
- Identity tokens, exchanged at the registry's OAuth token endpoint using a refresh-token grant. This case is not optional: an Azure Container Registry entry's plaintext auth field decodes to a null-GUID username with an empty password, so a client that only reads that field gets a 401 from what is likely the organisation's primary registry.

There is no setting for supplying a token by hand. This was explicitly rejected — the credentials already exist on disk and asking for them again is a configuration burden that solves nothing.

Anonymous requests are attempted **only** for a known set of public registries. Any other registry with no resolvable credential yields an unverifiable verdict carrying a "needs login" reason, which the UI surfaces.

### Existence checks

Checks issue a manifest `GET` against the distribution API, with an `Accept` header covering both OCI and Docker manifest and index media types, authenticated through the standard bearer-token challenge flow.

`GET` rather than `HEAD` specifically so that the 404 response body's error code distinguishes an unknown repository from an unknown manifest. A `HEAD` returns bare status and cannot separate the two, and separating them is a stated requirement.

At most six requests are in flight at once.

### Verdicts

Four outcomes, modelled explicitly rather than as a boolean plus an error:

- **Exists**, carrying the registry that matched.
- **Repository not found.**
- **Tag not found.**
- **Unverifiable**, carrying a reason: no credential, authentication failure, network failure, unsupported registry, or a downgraded fallback result.

The invariant that shapes the whole design: **unverifiable never produces a diagnostic.** An expired token must never look like a missing image.

A caveat is accepted rather than solved: some registries deliberately return 404 or 401 for private repositories the caller cannot see, so "does not exist" and "no access" are not always separable. Those land in unverifiable.

### Editor surfaces

- **Decoration** — a checkmark on references that exist. The matched registry is appended only when it differs from what the file names, so the annotation carries information rather than becoming wallpaper.
- **Diagnostic, error severity** — for the two not-found verdicts only. Messages name the repository or tag that is missing, name the chart metadata when the tag came from `appVersion`, and name the overriding registry when one redirected the check. That last point matters because the Problems panel strips away the inline decoration that would otherwise convey it.
- **Diagnostic, warning severity** — for a `latest` tag. This is a style rule, not a fact about a registry, and is kept at a lower severity so error severity continues to mean "this reference is broken". It is independent of the existence check, so a nonexistent `latest` produces both markers on the same range; no special-casing.
- **Hover** — the matched registry, `appVersion` provenance, and the reason behind an unverifiable verdict.
- **Status bar** — active override count, and count of registries needing a login.
- **Notification** — at most once per registry per session, when a credential is missing, offering to run the login command with the hostname filled in and to dismiss that registry permanently. Dismissals persist in extension state rather than in settings, because a dismissal is not configuration and should not accumulate in a checked-in settings file.

A missing credential deliberately produces **no** diagnostic. It is a fact about the developer's machine, not a defect in the file, and putting it in the Problems panel next to real errors trains people to ignore the panel.

### Sidebar

A tree view with a scope toggle in its title bar, switching between the current file and the whole workspace, defaulting to the current file. In workspace scope the top level lists values files, and expanding one is what triggers its checks — so scope can be widened without firing hundreds of requests at once.

Nodes reveal their line in the editor on click, and offer re-check at node, file, and root level.

### Settings

Two settings: a list of registry hostnames, and a boolean for the `latest` warning, defaulting to on.

The registry list is a plain array of strings and does a single job: non-empty it is the override set, empty it means fall back to the file's own registry. An earlier design had a separate "default registry" scalar alongside it; that was collapsed because the two differ only in the case where the file names no registry and the list is empty, which is already handled by the Docker Hub fallback rule.

Object-shaped entries with enable/disable flags were rejected — disabling is a comment-out in JSON. Per-image pattern routing was rejected as a different feature with a different mental model, to be shaped when a second real example exists rather than guessed at now.

User and workspace scopes are left to VS Code's native settings merge; the extension does not take a position on which one a developer should use.

No dedicated configuration UI ships in this spec. The settings JSON is the configuration surface.

### Caching and triggers

Checks run on document open and 750 milliseconds after typing stops. Save is not a trigger on its own — learning about a typo should not require committing to it.

Results persist across window reloads in extension state, with asymmetric expiry:

- **Exists** — long-lived, on the order of a day. A tag that resolved once is effectively immutable.
- **Not found** — expires in about a minute, because the developer is very likely about to push exactly that tag.
- **Unverifiable** — expires in tens of seconds.

This asymmetry is what makes the design affordable. Docker Hub counts manifest fetches against pull rate limits — roughly 100 per six hours anonymously per IP address, 200 authenticated on a free account — and the steady state of a green workspace must not spend that budget on every file open. With a long positive expiry, the cost is about one request per image per day.

A watcher on chart metadata files evicts that chart's cached results when `appVersion` changes, because otherwise a version bump leaves stale checkmarks at precisely the moment correctness matters most.

## Testing Decisions

A good test here asserts external behavior: given this values file source, these references come out; given this reference and this registry state, this verdict comes out. Tests should not reach into how normalization is layered, how the credential chain is ordered internally, or which helper function produced a result. Where a rule is invisible from the return value — which URL was requested, which credential was sent — the assertion belongs on the injected dependency's received calls, not on an internal function.

Four seams, chosen to be as high as possible and to keep logic testable without booting an editor or touching a network.

**Image reference extraction, in the Helm package.** Pure: values file source in, image references with source ranges out. Fixture strings in, expected references out, with no mocking of any kind. This seam covers structural detection and the corroborating-signal rule, the raw-source-text reading that keeps `1.10` from becoming `1.1`, template-syntax skipping, tagless references, and the document's own registry declaration.

**Chart context resolution, in the Helm package.** Filesystem reads injected as a dependency, so tests supply a directory shape rather than writing temporary files. Covers nearest-ancestor resolution, subcharts resolving against their own metadata, and absent metadata or absent `appVersion`.

**Existence checking, in the OCI registry package.** The single public entry point, with the fetch implementation, Docker configuration contents, and credential-helper subprocess runner all injected. Everything else is exercised through it by asserting the requests a fake fetch received and the verdict returned: normalization and `library/` prefixing, override fan-out and which registry matched, the Docker Hub downgrade, the whole credential chain including identity-token exchange, the bearer-token challenge flow, and the distinction between an unknown repository and an unknown manifest. Deliberately no separate seam for normalization or for credentials — they have no independent external behavior worth asserting.

**Extension wiring, through the existing entry point and the shared stub.** Prior art is already established and should be followed rather than reinvented: `apps/vscode/vitest.config.mts` aliases the `vscode` module to one checked-in stub, and its comment explicitly forbids per-file mock factories in favour of extending that stub. The existing activation test is the pattern. This seam covers registration — diagnostics collection, decoration type, tree provider, status bar item, watchers, commands — and none of the logic, all of which lives behind the three package seams above.

The other prior art in the repo, `apps/cli/tests/mct.spec.ts`, spawns the real built binary as a child process. That pattern is not applicable here because no binary ships in this spec, but it is the model to follow if a CLI is added later.

No integration test against a real registry. ADR 0001 records the deliberate exclusion of an end-to-end workspace, and standing up a live registry would contradict it; the injected fetch seam covers the protocol behavior that matters.

## Out of Scope

- **A CLI.** Both packages are built to be consumable by one, and neither imports `vscode`, but no CLI command ships here.
- **Digest-pinned references.** Supporting them means extra parsing of digest suffixes and digest sibling keys; they produce no marker.
- **Tag suggestions.** Listing a repository's tags to offer "did you mean 1.1.2" is genuinely useful and genuinely separate — tag listing is paginated, can be enormous, and is often denied to accounts that can pull.
- **Per-image registry routing.** Mapping specific images to specific registries is a different mental model from a flat override set.
- **Mutable-tag warnings beyond `latest`.** No configurable pattern list for `main`, `dev`, `stable` and similar until there is a second real example to shape it.
- **A dedicated registry configuration UI.** No command-palette flow, no webview. Settings JSON only.
- **Eager workspace scanning.** The tree checks on expansion; it does not check everything up front.
- **A setting for supplying registry tokens by hand.** Explicitly rejected.
- **Publishing either package.** Both stay private, consistent with ADR 0001.

## Further Notes

**This contradicts ADR 0001, and the contradiction should be resolved deliberately.** That ADR records that `packages/*` "is empty and stays empty until a second consumer actually needs shared logic — we don't pre-create placeholder packages." This spec creates two packages while the extension is the only consumer; the CLI that would be the second consumer is hypothetical.

The argument for proceeding anyway: the registry package is not a placeholder or a speculative extraction, it is a substantial body of logic — credential chain, token flows, protocol handling — that has nothing to do with Helm or with editors, and the repo has already committed to this split elsewhere. The vitest configuration in the extension workspace states that "real logic (never importing `vscode`) belongs in `packages/*`, where it needs no mocking at all", which is a testability argument for the seam independent of any second consumer. The argument against is simply that the ADR says not to. Worth reopening the ADR rather than silently overriding it.

**There is no `CONTEXT.md` in this repo yet,** so this spec had no glossary to draw on and introduces its own vocabulary: *image reference*, *values file*, *chart context*, *registry override set*, *verdict*, *unverifiable*. If these terms are worth keeping, they are the natural seed for a glossary.

**The single most important invariant** is that unverifiable never produces a diagnostic. Several decisions exist only to protect it — the four-way verdict model, the Docker Hub fallback downgrade, routing missing credentials to a notification instead of the Problems panel. A change that makes any uncertain outcome render as an error would defeat the feature, because a linter that cries wolf gets disabled and then none of the rest matters.

**Rate limits are a real design constraint, not a footnote.** The combination of a checkmark on every good image and a hard per-IP limit on Docker Hub manifest fetches is what forces the asymmetric cache expiry and the lazy tree expansion. Any future change that increases check frequency should be weighed against that budget.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the Problem Statement, Solution, and user stories, noting that no repository files or tests are named. The implementation is done when the reusable workspace packages and VS Code extension cover the specified Helm image discovery, registry verification, diagnostics, credentials, navigation, caching, and configuration behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, helm, typescript
Domain
devops, infrastructure, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.