microsoft / microsoft/GitHub-Copilot-for-Azure

Integration test failure: azure-deploy – todo list app service Terraform eastus2 [Assertion mismatch]

Open
#1,285 1 comment 0 reactions 1 assignee View on GitHub

@wbreza is already working on this.

Since Mar 16, 2026.

azure-deploy bug integration-test test-failure
Dominant language
Python
Stars
250
Forks
204
Avg merge
1d 12h
Merged PRs (30d)
67

Description

## Prompt

```
Create a todo list with frontend and API and deploy to Azure App Service using Terraform infrastructure in my current subscription in eastus2 region.
```

## Summary

**Run:** [Integration Tests - all #86](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23083939668)
**Test:** `azure-deploy_ - Integration Tests › terraform-app-service-deploy › creates todo list with frontend and API using Terraform`
**Result:** Fail (inconsistent — passed in app-service job, failed in terraform-app-service job)
**Duration:** 25m 53s (pass in app-service job) / 9m 55s (fail in terraform-app-service job)

## Root Cause Category

**Assertion mismatch**

## Diagnosis

### What was expected
The agent should create a todo list app with frontend and API, deploy to Azure App Service using Terraform in eastus2, and return an `*.azurewebsites.net` URL. Test expects `.tf` files in the workspace and no `.bicep` files.

### What actually happened
`containsDeployLinks = false` (line 442) in the dedicated `terraform-app-service-deploy` job. Notably, this same test **passed** in the `app-service-deploy` job (which runs both Bicep and Terraform variants together). The 9m 55s duration in the failing run vs 25m 53s in the passing run suggests the faster run may have short-circuited or failed earlier.

### Why it failed
This test shows inconsistent behavior: pass in one job context, fail in another. The very short 9m 55s duration in the failing run (vs 25m 53s passing) suggests the agent may have taken a different code path or the Terraform initialization failed quickly. Possible causes: timing-dependent resource availability, or different model responses when the test runs in isolation.

### Suggested fix
- Investigate why the test behaves differently when run in isolation vs. together with the Bicep variant.
- Check for Terraform state conflicts or initialization failures.

## azure-deploy Skill Invocation

| Skill | Invoked |
|-------|---------|
| **azure-deploy** | Unknown (inconsistent — passed in one job, failed in another) |

> In azure-deploy integration tests, the full skill chain is azure-prepare → azure-validate → azure-deploy.

## Skill Report Context

Per-test section from SKILL-REPORT.md (click to expand)

_Skill report not available — artifact download requires network access not available in this environment._

## Environment

- **Runner OS:** ubuntu-latest
- **Node.js:** 24.x
- **Model:** default (no MODEL_OVERRIDE set)
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23083939668
- **Commit:** c624dcaa1a9820f915eb4b0b8b3fb6e420d4815c

> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23087898435) · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.