microsoft / microsoft/GitHub-Copilot-for-Azure
Integration test failure: azure-quotas – quota check VM compute resources skill not invoked [Skill not invoked]
- Dominant language
- Python
- Stars
- 250
- Forks
- 204
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 67
Description
## Prompt
````
Check my Azure VM quota limits and current usage in East US
```
## Summary
**Run:** [Integration Tests - all #110](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23379277272)
**Test:** `azure-quotas_ - Integration Tests › azure-quotas › provides quota check commands for compute resources`
**Result:** Fail
**Duration:** 5m 50s
## Root Cause Category
**Skill not invoked**
## Diagnosis
### What was expected
The test expects all three of:
1. `isSkillUsed` (= `isSkillInvoked(agentMetadata, SKILL_NAME)`) = `true`
2. `mentionsQuotaCmd` = the agent response or tool call args include `az quota`
3. `mentionsScope` = the agent response or tool call args include `/subscriptions/`
### What actually happened
`isSkillUsed` was `false` — the `azure-quotas` skill was not invoked. The subsequent assertions (`mentionsQuotaCmd`, `mentionsScope`) also likely failed as a result.
```
> 115 | expect(isSkillUsed).toBe(true);
````
### Why it failed
The prompt "Check my Azure VM quota limits and current usage in East US" did not trigger the `azure-quotas` skill. The agent may have answered the quota question from general knowledge or used Azure CLI commands directly without routing through the skill. The skill's description may not strongly match "check quota limits and current usage" queries.
### Suggested fix
- Update `azure-quotas` skill description to include trigger phrases like "quota limits", "current usage", "VM quota", "vCPU limits"
- Verify the skill's "When to Use" section covers compute quota check queries by region
- Test whether the prompt is reaching the skill in live Copilot sessions
## azure-quotas Skill Invocation
| Skill | Invoked |
|-------|---------|
| **azure-quotas** | No |
## Skill Report Context
Skill report artifact not accessible (network policy blocks Azure Blob Storage downloads in the AWF sandbox). See run artifacts at https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23379277272/artifacts/6039510637
## Environment
- **Runner OS:** ubuntu-latest
- **Node.js:** v24.x
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23379277272
- **Commit:** `2f186f53859a62d458f62897cf089476ca1eb05d`
- **Test file:** `azure-quotas/integration.test.ts:115`
> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/23381600951) · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)
> [!WARNING]
>
> ⚠️ Firewall blocked 1 domain
>
> The following domain was blocked by the firewall during workflow execution:
>
> - `productionresultssa3.blob.core.windows.net`
>
> To allow these domains, add them to the `network.allowed` list in your workflow frontmatter:
>
> ```yaml
> network:
> allowed:
> - defaults
> - "productionresultssa3.blob.core.windows.net"
> ```
>
> See [Network Configuration](https://github.github.com/gh-aw/reference/network/) for more information.
>
>
Contributor guide
Assessment
This issue has not been assessed yet.