Stale `!terraform.output`/`!terraform.state` in downstream components in single apply with `--affected --include-dependents`
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 175
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 134
Description
### Describe the Bug
When running `atmos terraform apply --affected --include-dependents` and there's a change in the outputs of an upstream component, `!terraform.output` (and `!terraform.state`) values needed in downstream components are resolved and cached before any applies execute. The cache is not invalidated after an upstream component applies, so downstream dependents are applied with the pre-apply output values.
The apply command needs to be run a second time for it to pick up the updated output value in the dependent components.
### Expected Behavior
The command `atmos terraform apply --affected --include-dependents` should also apply changes to downstream components that depend on the updated output value of an upstream component.
### Steps to Reproduce
1. Components
```terraform
# components/terraform/networking/main.tf
resource "random_id" "vpc" {
byte_length = 4
}
# components/terraform/networking/outputs.tf
output "vpc_id" {
value = "vpc-${random_id.vpc.hex}"
}
# components/terraform/compute/main.tf
variable "vpc_id" { type = string }
resource "random_id" "cluster" {
byte_length = 4
keepers = {
vpc_id = var.vpc_id
}
}
```
2. Stack wiring
```terraform
# stacks/catalog/compute.yaml
components:
terraform:
compute:
backend_type: local
dependencies:
components:
- component: networking
vars:
vpc_id: !terraform.output networking vpc_id # also tried !terraform.state
# stacks/orgs/example.yaml
import:
- catalog/networking
- catalog/compute
vars:
stage: repro
```
3. Atmos config
```yaml
# atmos.yaml
base_path: "."
components:
terraform:
base_path: components/terraform
auto_generate_backend_file: true
init_run_reconfigure: false
stacks:
base_path: stacks
included_paths:
- "orgs/**/*"
name_pattern: "{stage}"
logs:
level: Info
```
4. Initial apply
```shell
atmos terraform apply networking -s repro -- -auto-approve
atmos terraform apply compute -s repro -- -auto-approve
git add . && git commit -m "initial"
```
Note the current vpc_id value.
5. Change networking's output
```terraform
# outputs.tf
output "vpc_id" {
value = "vpc-${random_id.vpc.hex}-v2" # added suffix
}
```
Commit the change.
6. Run the affected apply
```shell
atmos terraform apply -s repro --affected --include-dependents -- -auto-approve
```
7. Observed
- networking applies and its output changes to include -v2.
- compute applies but reports `No changes. Your infrastructure matches the configuration.`
- Compute's state shows `keepers.vpc_id` still holds the old value.
8. Re-run the same command
```shell
atmos terraform apply -s repro --affected --include-dependents -- -auto-approve
```
Now compute correctly sees the change and replaces the `random_id.cluster` resource with the new keeper value.
### Screenshots
_No response_
### Environment
_No response_
### Additional Context
_No response_
Contributor guide
Research direction
Start by tracing the `atmos terraform apply --affected --include-dependents` entry point, focusing on resolution and caching of `!terraform.output` and `!terraform.state`. Reproduce the networking and compute example, then verify that downstream values are refreshed after the upstream apply so one command updates both components.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, terraform
- Domain
- cli, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100