atmos vendor update silently skips re-fetching a component after bumping its version, when the version is encoded in the source URL's query string
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 175
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 134
Description
### Describe the Bug
`atmos vendor update` (and `--pull-request`) correctly detects a newer version and updates the `version:` field in `vendor.yaml`, but for sources that encode the git ref via a query string (e.g. `source: "github.com/org/repo.git//?ref={{.Version}}"`), the subsequent re-fetch is silently skipped. The vendor-lock drift check (`pkg/vendoring/lockfile.IsMaterialized`) decides the target is still "materialized" and never re-downloads, so the actual vendored files are never updated — only the `version:` string in `vendor.yaml` changes.
Root cause: `pkg/downloader/artifact.go`'s `RedactSource` — used to normalize/compare the "declared source" recorded in `vendor.lock.yaml` — unconditionally strips the URL's query string (`parsed.RawQuery = ""`) to redact embedded credentials before persisting/logging a source URL. But when a source encodes its version as a query parameter (`?ref=0.24.0` → `?ref=0.25.0`), redaction also erases the version, making both versions compare as identical to the lock's drift check (`pkg/vendoring/install/install.go`'s `FilterPending` → `isMaterialized`). Result: the lock incorrectly treats a real version bump as "no drift," and the download is skipped.
`?ref={{.Version}}` is the standard, documented way to version-pin a git-based vendor source in Atmos (it's used in `website/docs/cli/configuration/vendor.mdx`'s own examples, and in `examples/scaffolding`, `examples/demo-vendoring`, and `examples/demo-component-versions`), so this isn't a narrow edge case — it likely affects most real-world `atmos vendor update` usage against git sources.
### Expected Behavior
Bumping a source's version in `vendor.yaml` should always trigger a re-fetch of that component's files, regardless of whether the version is expressed via a query parameter (`?ref=`) or elsewhere in the source string.
### Steps to Reproduce
1. Add a `vendor.yaml` source using a query-string ref, e.g.:
```yaml
spec:
sources:
- component: "null-label"
source: "github.com/cloudposse/terraform-null-label.git//?ref={{.Version}}"
version: "0.24.0"
targets: ["components/terraform/null-label"]
included_paths: ["**/*.tf", "**/*.md"]
```
2. `atmos vendor pull` (seeds `vendor.lock.yaml` at 0.24.0).
3. `atmos vendor update --pull` (0.25.0 is available and gets discovered).
4. Observe: `vendor.yaml`'s `version:` field updates to `0.25.0`, but the files under `components/terraform/null-label` are byte-identical to before, and files added upstream between the two versions (e.g. a new `descriptors.tf` in this repro) never appear.
5. Debug logs (`ATMOS_LOGS_LEVEL=Debug`) show: `Vendor target matches immutable lock receipt; skipping download`.
### Environment
- Found while testing the not-yet-merged Azure DevOps pull-request provider (cloudposse/atmos#3048), but reproduces independently of that PR — this is a `pkg/vendoring/lockfile`/`pkg/downloader` bug, not specific to any pull-request provider.
- `atmos` built from `main` at commit `7f090db0f` (plus the unrelated PR #3048 branch, which doesn't touch this code path).
### Additional Context
Likely fix direction: `RedactSource`'s job is stripping credentials (userinfo, and presumably token-bearing query params), not the ref/version itself. Either exempt ref-like query params from redaction when computing the lock's comparison key, or track the declared version separately from the redacted source string so drift detection can independently ask "did the on-disk content change" and "did the declared version change."
Contributor guide
Research direction
Start with pkg/downloader/artifact.go's RedactSource and trace its recorded source into pkg/vendoring/lockfile and pkg/vendoring/install/install.go's FilterPending/isMaterialized logic. Reproduce with a vendor source using ?ref={{.Version}} and debug logging. Done means a version change in the query string is recognized as drift and the component is re-fetched, including newly added upstream files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100