Azure / Azure/azure-verified-modules-tools

Terraform testing guidance contradicts itself on `command = apply` for `expect_failures` tests

Open Beginner friendly
#76 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
PowerShell
Stars
2
Forks
5
Avg merge
12h 51m
Merged PRs (30d)
30

Description

The Terraform testing reference states a hard rule, then gives an example that cannot satisfy it.

The rule, in `src/Avm.Authoring/.../terraform-test.md` (mirrored into consuming repos as the `avm-tf-testing` skill's `references/terraform-test.md`):

> **Critical rule**: Unit tests use `command = apply` (NOT `command = plan`) because mocked providers make apply safe and allow testing resource creation logic.

And under "Testing Validation Rules":

```hcl
run "test_invalid_location_rejected" {
command = apply

variables {
location = ""
}

expect_failures = [var.location]
}
```

That example does not pass. A variable `validation` block fails during planning, so the apply never executes, and Terraform marks the whole run failed even though the expected failure did occur:

```
run "flex_consumption_requires_runtime_name"... fail
│ Warning: Expected failure while planning
│ A custom condition within var.fc1_runtime_name failed during the planning
│ stage and prevented the requested apply operation. While this was an
│ expected failure, the apply operation could not be executed and so the
│ overall test case will be marked as a failure...
Failure! 0 passed, 1 failed, 10 skipped.
```

The documented example was also reproduced **verbatim** in an isolated scratch module — a single `location` variable with the exact `validation` block and the exact `run` block from the reference — and it fails there too, on Terraform v1.15.8. So this is not an interaction with any particular module's configuration.

### Why this is worth fixing rather than living with

It is self-reinforcing. Anyone who writes a validation test, tries `apply`, watches it fail with a diagnostic that never names the real constraint, and reaches for `plan`, produces a file that quietly violates the stated rule with nothing explaining why.

We hit this concretely while adding unit tests to `Azure/terraform-azurerm-avm-res-web-site`: across eight pull requests, five of seven new test files used `command = plan`, and at least two sessions arrived there independently by exactly that route. A reviewer applying the "Critical rule" literally then asked a contributor to change a working suite to one that could not pass.

### Suggested fix

1. Add an explicit carve-out to the critical rule: `expect_failures` runs targeting variable-level `validation` blocks must use `command = plan`, because variable validation fails during planning and an unexecuted apply marks the run failed.
2. Correct the `test_invalid_location_rejected` example to use `command = plan`.

It is worth stating the boundary precisely, since `expect_failures` against a resource `precondition` or `postcondition` may behave differently from one against a variable `validation`.

_Drafted by Claude Opus 5._

Contributor guide

Open the contributing guide

Research direction

Start with src/Avm.Authoring/.../terraform-test.md and its mirrored references/terraform-test.md, then review the `expect_failures` example and reproduce it with Terraform v1.15.8. Done means the guidance distinguishes variable validation from other conditions and the invalid-location example uses the command that passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
terraform
Domain
devops, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.