atmos describe affected: nested YAML functions in cross-stack !terraform.state target resolve in caller's stack context instead of target's
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 175
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 134
Description
## Describe the Bug
When `atmos describe affected` evaluates a cross-stack `!terraform.state ` reference (3-arg form), atmos pulls the target component's merged YAML config and evaluates any nested YAML functions it contains. Those nested functions appear to resolve against the **caller's** stack context rather than the **target's** stack context, producing false `Could not find the component X in the stack Y` errors whenever a same-stack lookup in the target (e.g. `!terraform.state kms key_arn`) refers to a component that exists in the target's stack but not in the caller's.
This surfaces only when `ci.enabled: true` and the new zero-config CI base detection from #2241 are both in effect — i.e. when `describe affected --upload` is running without explicit `--ref` / `--sha` / `--base` and with function evaluation enabled (the default, equivalent to `--process-functions=true`).
## Expected Behavior
When atmos evaluates a nested `!terraform.state ` (two-arg, same-stack) call that appears inside a component whose outputs are being resolved cross-stack, the nested call should resolve against the component's **own** stack — not the caller's. In practice, that means:
- Caller stack: `stack-a` references `!terraform.state target-component stack-b output`.
- Target component in `stack-b` has `kms_key_id: !terraform.state kms key_arn` in its merged config.
- Resolving the outer 3-arg reference requires reading state from `stack-b`. If atmos walks the target's config to do that, any nested same-stack lookups should resolve against `stack-b` (where `kms` exists), not against `stack-a`.
## Current Behavior
Nested same-stack `!terraform.state ` calls inside the target's merged config are evaluated against the caller's stack context. When the referenced component doesn't exist in the caller's stack, `describe affected` fails with:
```
failed to describe component kms in stack in YAML function:
!terraform.state kms key_arn invalid component: Could not find the component kms in
the stack . Check that all the context variables are correctly defined
in the stack manifests. Are the component and stack names correct? Did you forget an
import?
```
The `caller-stack` in the error is the stack that initiated the describe-affected walk (e.g. a PR branch's affected-components output), not the stack the nested reference lives in.
## Minimal Reproduction
Two stacks and three components:
1. Component `kms` deployed **only** in stack `stack-b`.
2. Component `secrets-manager/shared` deployed in `stack-b`. Its merged config contains a same-stack reference: `kms_key_id: !terraform.state kms key_arn`.
3. Component `lambda/consumer` deployed in `stack-a`. Its config references the secrets-manager component cross-stack:
```yaml
CREDS_ARN: !terraform.state secrets-manager/shared stack-b secret_arns_map.shared/credentials
```
Run with `ci.enabled: true`:
```bash
atmos describe affected --upload
# or equivalently
atmos describe affected --upload --process-functions=true
```
On a PR that modifies the stack-wide config (e.g. an `_defaults.yaml` edit that makes `stack-a` appear as affected), atmos fails with:
```
failed to describe component kms in stack stack-a in YAML function:
!terraform.state kms key_arn
```
Even though `kms` is not referenced by any component in `stack-a` directly. The reference being evaluated is the nested one inside `secrets-manager/shared` (which lives in `stack-b` and does have `kms` co-located).
## Workaround
Pass `--process-functions=false` to `atmos describe affected`:
```bash
atmos describe affected --upload --process-functions=false
```
This skips all YAML function evaluation, which is acceptable for describe-affected (the command only needs stack-level change detection, not resolved values) but is a deviation from the canonical CI pattern shown at https://github.com/cloudposse-examples/atmos-pro-example-advanced where the flag is omitted.
## Environment
- atmos `v1.216.0` (also reproduces with `v1.215.0`)
- `ci.enabled: true` in the active profile
- `provider=github-actions`, `event=pull_request` (zero-config CI base detection, #2241)
- Workflow does `atmos describe affected --upload` against a PR where the affected-stacks set includes a stack whose components transitively reference a cross-stack component whose merged config contains a same-stack `!terraform.state` to a component not deployed in the caller's stack.
## Why This Matters
Any repo that uses the canonical atmos-pro workflow pattern *and* has cross-stack `!terraform.state` references where the target's config contains same-stack nested functions will hit this. The failure mode isn't immediately obvious from the error text (the error names a "missing" component that doesn't appear in the caller's stack config at all — it's only present in the target's).
The same pattern works fine at `terraform apply` time because terraform itself resolves `!terraform.state` cleanly via backend state reads without re-evaluating the target's YAML functions. The bug is specific to the describe-affected YAML evaluation path.
## Suggested Fix
When resolving a 3-arg cross-stack `!terraform.state`, scope any nested YAML function evaluation to the **target** stack's context (the stack named by the 2nd arg), not the stack that initiated the describe-affected walk.
Contributor guide
Research direction
Start with `atmos describe affected --upload` using the minimal two-stack reproduction and `--process-functions=true`. Trace the `!terraform.state` 3-argument evaluation path and its nested function handling. Done means nested same-stack lookups use the target stack context, while the described CI scenario succeeds without requiring `--process-functions=false`.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, go, terraform
- Domain
- cli, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100