microsoft / microsoft/GitHub-Copilot-for-Azure
Integration test failure: azure-enterprise-infra-planner – SAP backup Bicep generation [Timeout]
- Dominant language
- Python
- Stars
- 250
- Forks
- 204
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 67
Description
## Prompt
```
Spin up a policy-driven backup for SAP workloads, ensure encrypted and compressed backups, and provide audit logs for all recovery tests. Assume all defaults to make the plan.
```
## Summary
**Run:** [Integration Tests - all #151](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/24189676708)
**Test:** `azure-enterprise-infra-planner_ - Integration Tests › response-quality › generates Bicep files from approved plan`
**Result:** Fail (Timeout)
**Duration:** 20m 0s (exceeded 1200000 ms limit)
## Root Cause Category
**Timeout**
## Diagnosis
### What was expected
The test expects azure-enterprise-infra-planner to generate Bicep files for a SAP HANA backup infrastructure within the 20-minute time limit. The response-quality test validates the end-to-end workflow: infrastructure plan → Bicep generation → deployment.
### What actually happened
The agent successfully orchestrated a complete SAP HANA backup infrastructure deployment. It generated a detailed `infrastructure-plan.json` (23KB) with architectural reasoning, produced 7 modular Bicep files (747 lines total): managed identity, Key Vault with CMK encryption, Log Analytics workspace, Recovery Services Vault, backup policy with full/differential/incremental schedules, and diagnostic settings.
However, the test exceeded the 20-minute Jest timeout. The likely contributing factors:
- Initial deployment encountered an API schema error (`monitoringSettings.azureMonitorAlertSettings` not supported in API 2025-02-01), requiring fix-and-retry
- Bicep schema lookups via Azure MCP tools added latency
- Recovery Services Vault provisioning is inherently slow (backup policies require several minutes to apply)
### Why it failed
The full end-to-end workflow (plan → generate → validate → deploy → verify) for SAP HANA backup infrastructure is inherently longer than 20 minutes when combined with deployment retries. The initial API schema error that required automated correction (+1 retry) pushed the total execution time past the limit.
### Suggested fix
- Consider increasing the test timeout for this response-quality scenario (SAP backup infrastructure is more complex than typical scenarios)
- Pre-validate Recovery Services Vault API schema compatibility before generating Bicep to eliminate fix-and-retry cycles
- Add upfront `monitoringSettings` API version detection to avoid the known incompatibility with `2025-02-01`
## azure-enterprise-infra-planner Skill Invocation
| Skill | Invoked |
|-------|---------|
| **azure-enterprise-infra-planner** | **Yes** |
## Skill Report Context
Per-test section from SKILL-REPORT.md (click to expand)
**What Happened:**
The skill orchestrated a complete SAP HANA backup infrastructure deployment workflow. It generated a detailed `infrastructure-plan.json` with architectural reasoning, then produced 7 modular Bicep files (747 lines total). Initial deployment encountered an API schema error (`monitoringSettings` property not supported in API version 2025-02-01), which was immediately fixed by removing the unsupported property. Redeployment succeeded, provisioning the complete backup infrastructure to East US 2.
**❌ What Went Wrong:**
- Initial deployment failed due to `monitoringSettings.azureMonitorAlertSettings` property not supported in Recovery Services Vault API version 2025-02-01 (+1 retry)
- Bicep linter warnings for unused parameters and retention period size validation
> Skill report evaluator (confidence 88%) assessed this as passing based on deployment success, but the test timed out before completion.
## Environment
- **Runner OS:** ubuntu-latest
- **Model:** claude-sonnet-4.6
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/24189676708
- **Commit:** 9ffd31a469768cb094315a1eb30de31bc40149db
> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/24196812974/agentic_workflow) · ● 3M · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)
Contributor guide
Assessment
This issue has not been assessed yet.