microsoft / microsoft/GitHub-Copilot-for-Azure

Integration test failure: azure-deploy – discussion board App Service Bicep westus2 [Deployment failure]

Open
#2,036 0 comments 0 reactions 1 assignee Claimed by @tmeschter View on GitHub
azure-deploy bug integration-test test-failure
Dominant language
Python
Stars
250
Forks
204
Avg merge
1d 12h
Merged PRs (30d)
67

Description

## Prompt

````
Create a discussion board application and deploy to Azure App Service using my current subscription in westus2 region.
```

## Summary

**Run:** [Integration Tests - all #188](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/24880499835)
**Test:** `azure-deploy_ - Integration Tests › vanilla-app-service-deploy › creates discussion board`
**Result:** Fail — `expect(received).toBe(expected) // Object.is equality — Expected: true, Received: false` (line 238 of `integration.test.ts`)
**Duration:** 16m 34s

## Root Cause Category

**Deployment failure** — SQL Server Entra-only authentication blocked by service principal permissions gap

## Diagnosis

### What was expected
The agent should create a Node.js discussion board application and deploy it to Azure App Service in westus2 with Bicep IaC, returning an `*.azurewebsites.net` URL. Test asserts `containsDeployLinks === true` at line 238.

### What actually happened
The agent successfully executed the full skill chain: `azure-prepare` → `azure-quotas` → `azure-validate` → `azure-deploy`. The `azure-prepare` skill generated 13 production-ready files (Node.js/Express backend, Azure SQL Database schema, frontend UI, and full Bicep infrastructure-as-code). All 10 `azure-validate` pre-deployment checks passed. The `azure-deploy` skill ran `azd up` which created all Azure resources (Resource Group, App Service, SQL Database, Application Insights). However, during provisioning, SQL Server configuration attempted to set Entra-only authentication but failed with:

```
InvalidExternalAdministratorSid: Invalid external administrator object id
````

The service principal used in the CI/CD test environment lacks Microsoft Graph API permissions required to be designated as SQL Server Entra admin. This caused `azd up` to fail after infrastructure was partially provisioned, without returning a deployment URL. The test assertion for `containsDeployLinks` failed as a result.

### Root causes
1. **SQL Server Entra-only authentication incompatible with service principal**: The generated Bicep template configured SQL Server with Entra-only auth (`azureADOnlyAuthentication: true`) but the CI/CD service principal lacks `User.Read.All` / `GroupMember.Read.All` Microsoft Graph permissions required to be designated as AAD admin.
2. **No fallback to SQL authentication**: When Entra auth fails, the skill has no fallback to SQL username/password (stored in Key Vault), causing deployment to halt entirely.
3. **Deploy link not emitted**: Because `azd up` exited with error, no `*.azurewebsites.net` URL was produced, causing `containsDeployLinks` assertion to fail.

### Suggested fix
- Add detection logic in azure-prepare (or azure-deploy) to check if the executing identity is a service principal and select SQL authentication instead of Entra-only auth
- Alternatively, generate Bicep with dual-auth support (Entra + SQL fallback) and store SQL credentials in Key Vault
- Update azure-deploy error recovery to detect `InvalidExternalAdministratorSid` and automatically re-provision with SQL auth mode

## azure-deploy Skill Invocation

| Skill | Invoked |
|-------|---------|
| **azure-prepare** | ✅ Yes |
| **azure-validate** | ✅ Yes |
| **azure-deploy** | ✅ Yes |

> In azure-deploy integration tests, the full skill chain is azure-prepare → azure-validate → azure-deploy. The deployment was invoked but failed due to SQL auth incompatibility with the CI/CD service principal.

## Skill Report Context

From the `vanilla-app-service` SKILL-REPORT (run 2026-04-24):

- **Overall Test Pass Rate (agent-reported):** 50% (1/2) — agent reported Test 1 (discussion board) as PARTIAL PASS with 85% confidence
- **Average Confidence:** 75%
- The SKILL-REPORT marks this as passing because infrastructure was generated and deployment was attempted. However, SQL Server Entra auth failure prevented `azd up` from completing and the deploy URL was never returned.
- Agent correctly identified the SQL auth error as a CI/CD environment limitation not affecting end users.

**Key agent findings:**
- ✅ All 13 application files generated successfully
- ✅ All 10 validation checks passed
- ✅ Azure resources provisioned (App Service, SQL Server, SQL DB, Application Insights)
- ❌ `azd up` exited with `InvalidExternalAdministratorSid` during SQL Server admin assignment
- ❌ No deployment URL returned → `containsDeployLinks = false`

## Environment

- **Runner OS:** ubuntu-latest
- **Model:** claude-sonnet-4.6
- **Region:** westus2
- **IaC:** Bicep (AZD)
- **Service:** Azure App Service + SQL Database + Application Insights
- **Run URL:** https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/24880499835
- **Commit:** aa0e1446f9138bf253b72b8ab154b382f3d15fee

> Generated by [Analyze Test Run](https://github.com/microsoft/GitHub-Copilot-for-Azure/actions/runs/24887948672/agentic_workflow) · ● 5.8M · [◷](https://github.com/search?q=repo%3Amicrosoft%2FGitHub-Copilot-for-Azure+is%3Aissue+%22gh-aw-workflow-call-id%3A+microsoft%2FGitHub-Copilot-for-Azure%2Fanalyze-test-run%22&type=issues)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.