DAG masking omits Terraform inputs for post-hooks
- Dominant language
- Go
- Stars
- 1.4k
- Forks
- 175
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 134
Description
## Summary
With masking enabled, Atmos DAG execution removes selected values from the generated Terraform JSON var-file, apparently to use the documented secret-safe `TF_VAR_` delivery path instead.
During post-hook execution, the corresponding value is not available to the child Terraform/OpenTofu process. The required input is absent from the var-file, causing Terraform/OpenTofu to report it as unset or to parse a malformed partial value.
The same DAG succeeds with `--mask=false`, which keeps the required values in the generated `.tfvars.json` file.
## Expected behavior
Masking must not alter the effective Terraform/OpenTofu input received by a component or post-hook.
When masking omits a value from a generated var-file, Atmos must deliver the equivalent value to every child Terraform/OpenTofu process that requires it, including processes invoked through DAG execution and post-hooks.
Atmos documents masking as output interception and documents secret-safe `TF_VAR_` delivery for automatically registered sensitive Terraform/OpenTofu outputs:
- https://atmos.tools/cli/configuration/settings/mask/#how-it-works
- https://atmos.tools/cli/configuration/settings/mask/#disable-masking-for-debugging
A direct component apply and an equivalent DAG apply should receive identical effective Terraform/OpenTofu inputs regardless of whether output masking is enabled.
## Actual behavior
With masking enabled, DAG execution produces an incomplete Terraform JSON var-file for a post-hook.
Comparing the masking-enabled file with the `--mask=false` file shows that required top-level fields are absent when masking is enabled, including application identity, deployment metadata, tags, hook-specific configuration, a secret-name prefix, and a required multiline payload.
The omitted multiline payload explains the Terraform/OpenTofu error that the variable is not set. In other cases, partially delivered multiline values are parsed as HCL and fail on YAML syntax.
Terraform/OpenTofu reports errors such as:
```text
Error: Extra characters after expression
Error: Unsupported operator
Error: No value for required variable
```
The affected hook consumes non-sensitive configuration. A separate sensitive output from the same component is not directly referenced by the hook, but appears to trigger the masking behavior.
The same component succeeds when:
1. Applied directly.
2. Applied through the DAG with `--mask=false`.
The documented `settings.terminal.mask.enabled: false` setting did not fully disable this behavior during DAG execution. Only the `--mask=false` CLI flag allowed the DAG to complete.
## Steps to reproduce
1. Configure a Terraform component with:
- A non-sensitive multiline string value consumed by a post-hook.
- A separate sensitive output.
2. Configure an Atmos post-hook that consumes only the non-sensitive value.
3. Include the component and hook in an Atmos dependency graph.
4. Run the component directly with masking enabled.
- Expected and observed: succeeds.
5. Run the dependency graph with masking enabled.
- Observed: the generated Terraform JSON var-file is missing required fields and the post-hook fails.
6. Run the same DAG command with `--mask=false`.
- Expected and observed: succeeds.
7. Compare the generated JSON var-files from both runs.
- Observed: required top-level fields are absent from the masking-enabled file.
## Multiline structured values
The failing value is a large multiline structured string. This may be an important trigger: if it is delivered through a Terraform/OpenTofu CLI-variable path rather than a JSON var-file, it can be parsed as HCL instead of preserved as a string.
The masking-enabled DAG/post-hook path may therefore have two related failures:
1. A value omitted from the JSON var-file is not propagated to the child process through `TF_VAR_`.
2. When a partial or fallback CLI-variable path is used, multiline structured content is parsed as HCL and fails.
A short scalar value may present only as an unset variable, while a multiline structured value makes the corruption visible through parser errors.
## Impact
This is not only a log-redaction issue. It can omit or invalidate Terraform/OpenTofu inputs used by post-hooks, preventing otherwise valid components from deploying through Atmos DAG execution.
Contributor guide
Research direction
Reproduce the component through direct apply and DAG execution with masking enabled, then compare the generated Terraform JSON var-files against the --mask=false run. Trace DAG post-hook child-process input propagation, including TF_VAR_ handling and multiline values; done means masked and unmasked runs deliver identical effective inputs and the post-hook succeeds.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, terraform
- Domain
- cli, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100