cloudposse / cloudposse/atmos

atmos vendor update silently skips re-fetching a component after bumping its version, when the version is encoded in the source URL's query string

Open
#3,076 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1.4k
Forks
175
Avg merge
2d 3h
Merged PRs (30d)
134

Description

### Describe the Bug
`atmos vendor update` (and `--pull-request`) correctly detects a newer version and updates the `version:` field in `vendor.yaml`, but for sources that encode the git ref via a query string (e.g. `source: "github.com/org/repo.git//?ref={{.Version}}"`), the subsequent re-fetch is silently skipped. The vendor-lock drift check (`pkg/vendoring/lockfile.IsMaterialized`) decides the target is still "materialized" and never re-downloads, so the actual vendored files are never updated — only the `version:` string in `vendor.yaml` changes.

Root cause: `pkg/downloader/artifact.go`'s `RedactSource` — used to normalize/compare the "declared source" recorded in `vendor.lock.yaml` — unconditionally strips the URL's query string (`parsed.RawQuery = ""`) to redact embedded credentials before persisting/logging a source URL. But when a source encodes its version as a query parameter (`?ref=0.24.0` → `?ref=0.25.0`), redaction also erases the version, making both versions compare as identical to the lock's drift check (`pkg/vendoring/install/install.go`'s `FilterPending` → `isMaterialized`). Result: the lock incorrectly treats a real version bump as "no drift," and the download is skipped.

`?ref={{.Version}}` is the standard, documented way to version-pin a git-based vendor source in Atmos (it's used in `website/docs/cli/configuration/vendor.mdx`'s own examples, and in `examples/scaffolding`, `examples/demo-vendoring`, and `examples/demo-component-versions`), so this isn't a narrow edge case — it likely affects most real-world `atmos vendor update` usage against git sources.

### Expected Behavior
Bumping a source's version in `vendor.yaml` should always trigger a re-fetch of that component's files, regardless of whether the version is expressed via a query parameter (`?ref=`) or elsewhere in the source string.

### Steps to Reproduce
1. Add a `vendor.yaml` source using a query-string ref, e.g.:
```yaml
spec:
sources:
- component: "null-label"
source: "github.com/cloudposse/terraform-null-label.git//?ref={{.Version}}"
version: "0.24.0"
targets: ["components/terraform/null-label"]
included_paths: ["**/*.tf", "**/*.md"]
```
2. `atmos vendor pull` (seeds `vendor.lock.yaml` at 0.24.0).
3. `atmos vendor update --pull` (0.25.0 is available and gets discovered).
4. Observe: `vendor.yaml`'s `version:` field updates to `0.25.0`, but the files under `components/terraform/null-label` are byte-identical to before, and files added upstream between the two versions (e.g. a new `descriptors.tf` in this repro) never appear.
5. Debug logs (`ATMOS_LOGS_LEVEL=Debug`) show: `Vendor target matches immutable lock receipt; skipping download`.

### Environment
- Found while testing the not-yet-merged Azure DevOps pull-request provider (cloudposse/atmos#3048), but reproduces independently of that PR — this is a `pkg/vendoring/lockfile`/`pkg/downloader` bug, not specific to any pull-request provider.
- `atmos` built from `main` at commit `7f090db0f` (plus the unrelated PR #3048 branch, which doesn't touch this code path).

### Additional Context
Likely fix direction: `RedactSource`'s job is stripping credentials (userinfo, and presumably token-bearing query params), not the ref/version itself. Either exempt ref-like query params from redaction when computing the lock's comparison key, or track the declared version separately from the redacted source string so drift detection can independently ask "did the on-disk content change" and "did the declared version change."

Contributor guide

Open the contributing guide

Research direction

Start with pkg/downloader/artifact.go's RedactSource and trace its recorded source into pkg/vendoring/lockfile and pkg/vendoring/install/install.go's FilterPending/isMaterialized logic. Reproduce with a vendor source using ?ref={{.Version}} and debug logging. Done means a version change in the query string is recognized as drift and the component is re-fetched, including newly added upstream files.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.