microsoft / microsoft/GitHub-Copilot-for-Azure

Integration test failure: azure-deploy – event-driven function app Terraform [Assertion mismatch]

Open
#1,655 0 comments 0 reactions 1 assignee Claimed by @paulyuk View on GitHub
azure-deploy bug integration-test
Dominant language
Python
Stars
250
Forks
204
Avg merge
1d 12h
Merged PRs (30d)
67

Description

## Prompt

```
Create an event-driven function app to process messages and deploy to Azure Functions using Terraform infrastructure in my current subscription in eastus2 region.
```

Follow-up: `Go with recommended options and proceed with Azure deployment.`

## Summary

**Run:** [Integration Tests - all #134](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23891189563)
**Test:** `azure-deploy_ - Integration Tests › terraform-azure-functions-deploy › creates event-driven function app with Terraform`
**Result:** Fail
**Duration:** 25m 17s (assertion mismatch — `containsDeployLinks` expected `true`, got `false`)

## Root Cause Category

**Assertion mismatch** — deployment did not complete within the test window; no deploy links generated

## Diagnosis

### What was expected
The agent should create an event-driven Azure Functions app with a Service Bus trigger using Terraform infrastructure, deploy to Azure, and return deploy links with `.tf` files in `infra/` and no `.bicep` files.

### What actually happened
The agent invoked only `azure-prepare`, which successfully generated a complete event-driven function app with Service Bus trigger (Python code, v2 decorator model, Terraform infrastructure files — 11 files total). The expected workflow continuation to `azure-validate` and `azure-deploy` skills never occurred. Instead, the agent bypassed the skill chain and directly executed bash commands:

```bash
terraform validate
terraform apply
```

Two Terraform configuration errors were discovered and fixed inline:
1. `Unsupported argument: enable_partitioning in azurerm_servicebus_queue`
2. `Unsupported block: function_app_config in azurerm_linux_function_app`

The deployment was initiated but did not complete within the ~25-minute test window, leaving no deploy links in the response.

### Root cause
1. **Skill chain not invoked**: After `azure-prepare` completed, the agent bypassed `azure-validate` and `azure-deploy` skills, running `terraform apply` directly via bash. This is inconsistent with the expected 3-skill chain pattern.
2. **Terraform configuration errors**: The generated infrastructure had unsupported arguments (`enable_partitioning`, `function_app_config`) causing initial `terraform validate` failures and requiring multiple correction cycles.
3. **Deployment timeout**: Terraform Azure Functions deployments can take 10-15+ minutes; combined with the error correction overhead (~25m total), the deployment exceeded the test window before generating a live URL.

### Suggested fix
- Investigate why `azure-validate` and `azure-deploy` skills are bypassed when `azure-prepare` generates Terraform infrastructure for event-driven Azure Functions
- Fix the Terraform templates for `azurerm_servicebus_queue` (remove `enable_partitioning`) and `azurerm_linux_function_app` (remove `function_app_config` block) to eliminate validation failures that slow down deployments
- Consider adding a check in `azure-prepare` that validates generated Terraform against current provider schema before handing off to `azure-validate`

## azure-deploy Skill Invocation

| Skill | Invoked |
|-------|---------|
| **azure-prepare** | ✅ Yes |
| **azure-validate** | ❌ No (skipped — agent ran terraform directly) |
| **azure-deploy** | ❌ No (skipped — agent ran terraform directly) |

> In azure-deploy integration tests, the full skill chain is azure-prepare → azure-validate → azure-deploy.

## Skill Report Context

From the `tf-functions` SKILL-REPORT (test run 2026-04-02):

- **Skill Invocation Success Rate:** N/A
- **Overall Test Pass Rate:** 66.67% (2/3)
- **Average Confidence:** 78.33%
- Event-driven test confidence: **65% (MEDIUM)**
- Key warning: "Expected skills (azure-deploy, azure-validate) were not invoked" and "Terraform deployment exceeded test time window"

Terraform errors caught and fixed during run:
- `Unsupported argument: enable_partitioning in azurerm_servicebus_queue`
- `Unsupported block: function_app_config in azurerm_linux_function_app`

## Environment

- **Runner OS:** ubuntu-latest
- **Node.js:** v22.22.0
- **Model:** claude-sonnet-4.5
- **Region:** eastus2
- **IaC:** Terraform
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23891189563
- **Commit:** dc149f6821c954cedf9045a0c88fcf7571f6b35c

> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23901135657) · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.