microsoft / microsoft/GitHub-Copilot-for-Azure
Integration test failure: azure-deploy – todo list app service Terraform eastus2 [Assertion mismatch]
@wbreza is already working on this.
Since Mar 16, 2026.
- Dominant language
- Python
- Stars
- 250
- Forks
- 204
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 67
Description
## Prompt
```
Create a todo list with frontend and API and deploy to Azure App Service using Terraform infrastructure in my current subscription in eastus2 region.
```
## Summary
**Run:** [Integration Tests - all #86](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23083939668)
**Test:** `azure-deploy_ - Integration Tests › terraform-app-service-deploy › creates todo list with frontend and API using Terraform`
**Result:** Fail (inconsistent — passed in app-service job, failed in terraform-app-service job)
**Duration:** 25m 53s (pass in app-service job) / 9m 55s (fail in terraform-app-service job)
## Root Cause Category
**Assertion mismatch**
## Diagnosis
### What was expected
The agent should create a todo list app with frontend and API, deploy to Azure App Service using Terraform in eastus2, and return an `*.azurewebsites.net` URL. Test expects `.tf` files in the workspace and no `.bicep` files.
### What actually happened
`containsDeployLinks = false` (line 442) in the dedicated `terraform-app-service-deploy` job. Notably, this same test **passed** in the `app-service-deploy` job (which runs both Bicep and Terraform variants together). The 9m 55s duration in the failing run vs 25m 53s in the passing run suggests the faster run may have short-circuited or failed earlier.
### Why it failed
This test shows inconsistent behavior: pass in one job context, fail in another. The very short 9m 55s duration in the failing run (vs 25m 53s passing) suggests the agent may have taken a different code path or the Terraform initialization failed quickly. Possible causes: timing-dependent resource availability, or different model responses when the test runs in isolation.
### Suggested fix
- Investigate why the test behaves differently when run in isolation vs. together with the Bicep variant.
- Check for Terraform state conflicts or initialization failures.
## azure-deploy Skill Invocation
| Skill | Invoked |
|-------|---------|
| **azure-deploy** | Unknown (inconsistent — passed in one job, failed in another) |
> In azure-deploy integration tests, the full skill chain is azure-prepare → azure-validate → azure-deploy.
## Skill Report Context
Per-test section from SKILL-REPORT.md (click to expand)
_Skill report not available — artifact download requires network access not available in this environment._
## Environment
- **Runner OS:** ubuntu-latest
- **Node.js:** 24.x
- **Model:** default (no MODEL_OVERRIDE set)
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23083939668
- **Commit:** c624dcaa1a9820f915eb4b0b8b3fb6e420d4815c
> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23087898435) · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.