microsoft / microsoft/GitHub-Copilot-for-Azure

Integration test failure: azure-prepare – static whiteboard web app plan not ready [Assertion mismatch]

Open
#1,268 2 comments 0 reactions 1 assignee Claimed by @wbreza View on GitHub
bug deploy skills test-failure
Dominant language
Python
Stars
250
Forks
204
Avg merge
1d 12h
Merged PRs (30d)
67

Description

## Prompt

```
Create a static whiteboard web app and deploy to Azure.
```

Follow-up: `Go with recommended options and proceed with Azure deployment.`

## Summary

**Run:** [Integration Tests - all #84](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23050118229)
**Test:** `azure-prepare_ - Integration Tests › prepare-deployment › creates project files for static whiteboard web app before validation`
**Result:** Fail
**Duration:** 3m 32s

## Root Cause Category

**Assertion mismatch**

## Diagnosis

### What was expected
The test expects: (1) `azure-prepare` skill is invoked, (2) `hasPlanReadyForValidation(agentMetadata)` returns `true` — i.e., a `plan.md` or equivalent deployment plan was created before validation begins.

### What actually happened
`isSkillInvoked(agentMetadata, SKILL_NAME)` passed (skill was invoked), but `expect(planReady).toBe(true)` failed — `hasPlanReadyForValidation` returned `false`. The skill ran but did not produce a plan that was recognized as ready for validation.

### Why it failed
The `azure-prepare` skill was invoked but the deployment plan was either (1) not created in the expected format, (2) created under a different path than `hasPlanReadyForValidation` checks, or (3) the static whiteboard app scenario uses a template that does not emit a `plan.md` file. The early-termination condition was also not met.

### Suggested fix
- Verify that `azure-prepare` writes a `plan.md` for static web app scenarios with the expected structure
- Update `hasPlanReadyForValidation` to cover the output format for this app type, or update the skill to emit a consistent plan file

## azure-prepare Skill Invocation

| Skill | Invoked |
|-------|---------|
| **azure-prepare** | **Yes** |

## Skill Report Context

Per-test section from SKILL-REPORT.md (click to expand)

_Skill report artifact was not accessible in this analysis run. See run artifacts at https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23050118229/artifacts/5911858622_

## Environment

- **Runner OS:** ubuntu-latest
- **Node.js:** v24.14.0
- **Model:** claude-sonnet-4.5 (or as overridden)
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23050118229
- **Commit:** `fef6de1232d9a1725537d8058e39e382e78233ab`

> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23055952855) · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.