microsoft / microsoft/GitHub-Copilot-for-Azure
Integration test failure: azure-validate – containerized Container Apps deployment [Skill not invoked]
@JasonYeMSFT is already working on this.
Since Apr 30, 2026.
- Dominant language
- Python
- Stars
- 250
- Forks
- 204
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 67
Description
## Prompt
```
Create a containerized web application and deploy to Azure Container Apps.
```
## Summary
**Run:** [Integration Tests - all #198](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/25108826699)
**Test:** `azure-validate_ - Integration Tests › deployment-validation › terminates at validation for containerized web app on Container Apps`
**Result:** Fail
**Duration:** 1m 12s
## Root Cause Category
**Skill not invoked**
## Diagnosis
### What was expected
The test asserts two things:
1. `deploymentCommandRan` should be `false` — meaning `azd up` / `azd deploy` should **not** run before validation
2. `validateInvoked || validationCommandRan` should be `true` — azure-validate should be invoked (or a validation command run) before deployment
The test expects the agent to invoke azure-prepare (prepare code/infra) → invoke azure-validate (validate before deploying) → stop at that point.
### What actually happened
The agent correctly invoked `azure-prepare` first and generated a complete TypeScript/Node.js application (Express.js framework), complete with a multi-stage Dockerfile, Bicep infrastructure templates, Application Insights monitoring, health probes, and auto-scaling configuration. The agent then immediately ran `azd up` to begin deployment — bypassing the azure-validate skill entirely.
Failure at `tests/azure-validate/integration.test.ts:214`:
```
Expected: false
Received: true
// deploymentCommandRan = true because agent ran `azd up`
````
### Why it failed
The `azure-prepare` skill's workflow ran `azd up` directly from within its flow, without pausing to invoke `azure-validate`. The azure-prepare skill's deployment plan is expected to include a validation phase (invoking azure-validate) before executing deployment. This failure matches the same root cause as the related static whiteboard issue (#2054): azure-prepare proceeds straight to `azd up` instead of routing to azure-validate first.
### Suggested fix
- Update the azure-prepare skill workflow to explicitly invoke `azure-validate` **before** running any `azd up` / `azd deploy` commands when deployment is requested.
- Alternatively, add a guard in azure-prepare that prevents running deployment commands without completing a validation step first.
## azure-validate Skill Invocation
| Skill | Invoked |
|-------|---------|
| **azure-validate** | No |
The agent invoked azure-prepare correctly, but azure-prepare ran `azd up` directly without chaining to azure-validate.
## Skill Report Context
Per-test section from SKILL-REPORT.md (click to expand)
### Test 3: deployment-validation_terminates_at_validation_for_containerized_web_app_on_Container_Apps
**Type:** End-to-End
**Prompt:** "Create a containerized web application and deploy to Azure Container Apps."
**Runs:** 1 | **Pass Rate:** 100% (per skill report) | **Avg Confidence:** 95%
**What Happened:**
The agent invoked azure-prepare and generated a complete TypeScript/Node.js application with Express.js framework. All deployment artifacts were created including multi-stage Dockerfile, Bicep infrastructure templates, and azure.yaml configuration. The agent initiated `azd up` to begin deployment.
**✅ What Went Well:**
- azure-prepare skill correctly identified and invoked immediately
- Complete application stack generated with production-ready patterns (Node.js 20, TypeScript, multi-stage Docker build)
- Comprehensive Bicep infrastructure with Application Insights monitoring
- Managed identity security model applied throughout
- Auto-scaling configured (1-3 replicas)
**❌ What Went Wrong:**
- Deployment command (`azd up`) ran before azure-validate was invoked
- `deploymentCommandRan` assertion failed (expected `false`, received `true`)
> Note: The SKILL-REPORT assessed this as 100% pass (soft-pass based on task completion), but the JUnit hard assertion `expect(deploymentCommandRan).toBe(false)` failed because `azd up` ran.
## Environment
- **Runner OS:** ubuntu-latest
- **Model:** claude-sonnet-4.5
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/25108826699
- **Commit:** `13345dedd7cc6cde2b1f9fc25e829bfa4b648071` (Add entra-agent-id skill)
> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/25119882862/agentic_workflow) · ● 5.7M · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.