Azure / Azure/azure-rest-api-specs
[BUG] ChatCompletionSkill PUT fails validation on 2026-04-01
- Dominant language
- TypeSpec
- Stars
- 3.1k
- Forks
- 5.9k
- Avg merge
- 2d 22h
- Merged PRs (30d)
- 444
Description
### API Spec link
https://github.com/Azure/azure-rest-api-specs/blob/main/specification/search/data-plane/Search/stable/2026-04-01/search.json
### API Spec version
2026-04-01
### Describe the bug
Creating or updating an Azure AI Search skillset that contains `#Microsoft.Skills.Custom.ChatCompletionSkill` via the skillset PUT endpoint fails with HTTP 400 and the message:
> `The API key provided for endpoint 'https://.openai.azure.com/openai/v1/chat/completions' is invalid or has been revoked.`
The spec permits the submitted configuration (the body conforms to the `ChatCompletionSkill` schema), and the target Azure OpenAI / AI Foundry endpoint accepts the same `apiKey` when called directly (HTTP 200). The failure is exclusive to the Search skillset validator executed during PUT. The same skillset body, with the `ChatCompletionSkill` removed, is accepted by the service — so the defect is isolated to the eager validation path of `ChatCompletionSkill`.
The bug reproduces across:
- Both REST API versions (`2025-11-01-preview`, `2026-04-01`)
- Both supported URI formats (`/openai/v1/chat/completions`, `/openai/deployments/{name}/chat/completions`)
- All three documented auth mechanisms:
- `apiKey` (fresh key rotated and verified via direct `curl`)
- `authIdentity` with user-assigned MI
- Implicit system-assigned MI (both `apiKey` and `authIdentity` omitted / `null`)
- Setting `authIdentity` to `#Microsoft.Azure.Search.DataNoneIdentity` → returns HTTP 500 instead
- Both REST (`az rest`) and Azure Portal skillset editor
As a practical consequence, once such a skillset is deleted, **it cannot be recreated** via any public API surface, because every well-formed body containing `ChatCompletionSkill` is rejected at validation.
### Expected behavior
Per spec, a PUT request with a body containing a properly shaped `ChatCompletionSkill` and valid `apiKey` (or a correctly configured `authIdentity` / system-assigned MI with required RBAC roles on the target resource) should succeed with HTTP 201/200. Eager validation — if performed — should use the auth material declared in the request body and should accept a valid key that is demonstrably functional at the target endpoint.
Equivalent submissions for `#Microsoft.Skills.Text.AzureOpenAIEmbeddingSkill` on the same skillset against the same AI Foundry resource with the same auth settings are accepted, confirming the declared auth/RBAC posture is correct.
### Actual behavior
PUT returns HTTP 400:
```json
{
"error": {
"code": "",
"message": "The API key provided for endpoint 'https://.openai.azure.com/openai/v1/chat/completions' is invalid or has been revoked."
}
}
```
The same URL and key used out-of-band return HTTP 200:
```
$ curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
-X POST 'https://.openai.azure.com/openai/v1/chat/completions' \
-H 'api-key: ' \
-H 'Content-Type: application/json' \
-d '{"model":"gpt-5.2","messages":[{"role":"user","content":"hi"}]}'
HTTP 200
```
Summary of attempts (all return the identical 400 message, unless noted):
| # | `apiKey` | `authIdentity` | `uri` path | API version | Result |
|---|---|---|---|---|---|
| 1 | omitted | omitted | `/openai/v1/chat/completions` | `2025-11-01-preview` | 400 |
| 2 | `null` | `null` (explicit) | `/openai/v1/chat/completions` | `2025-11-01-preview` | 400 |
| 3 | `null` | `DataNoneIdentity` | `/openai/v1/chat/completions` | `2025-11-01-preview` | **500** |
| 4 | `null` | UAMI | `/openai/v1/chat/completions` | `2025-11-01-preview` | 400 |
| 5 | `null` | UAMI | `/openai/v1/chat/completions` | `2026-04-01` | 400 |
| 6 | valid key (curl HTTP 200 confirmed) | omitted | `/openai/v1/chat/completions` | `2025-11-01-preview` | 400 |
| 7 | valid key | omitted | `/openai/deployments/gpt-5.2/chat/completions` | `2025-11-01-preview` | 400 |
| 8 | valid key | omitted | `/openai/deployments/gpt-5.2/chat/completions` | `2026-04-01` | 400 |
Removing `ChatCompletionSkill` entirely (keeping `ContentUnderstandingSkill` + `AzureOpenAIEmbeddingSkill` with `cognitiveServices` = `#Microsoft.Azure.Search.AIServicesByIdentity` + `identity: null`) → **PUT succeeds**. This isolates the regression to `ChatCompletionSkill`.
### Reproduction Steps
### Prerequisites
- An Azure AI Search service (any tier, Japan East verified)
- An Azure AI Foundry resource (kind `AIServices`) in an adjacent region, with a `gpt-5.2` chat deployment (or substitute any current chat model)
- A valid admin key or AAD token with data-plane access to the Search service
### Step 1 — Verify the target endpoint and key are healthy (out-of-band)
```bash
FOUNDRY_KEY=$(az cognitiveservices account keys list \
--name --resource-group --query key1 -o tsv)
curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
-X POST "https://.openai.azure.com/openai/v1/chat/completions" \
-H "api-key: $FOUNDRY_KEY" \
-H 'Content-Type: application/json' \
-d '{"model":"gpt-5.2","messages":[{"role":"user","content":"hi"}]}'
# Expected: HTTP 200
```
### Step 2 — Attempt skillset PUT with the same key
```bash
cat > /tmp/repro.json <.openai.azure.com/openai/v1/chat/completions",
"apiKey": "$FOUNDRY_KEY",
"inputs": [
{ "name": "systemMessage", "source": "='You are helpful.'" },
{ "name": "userMessage", "source": "='Hi'" }
],
"outputs": [ { "name": "response", "targetName": "out" } ],
"commonModelParameters": { "model": "gpt-5.2" }
}
]
}
EOF
az rest --method put \
--uri "https://.search.windows.net/skillsets/repro-chatcompletion-bug?api-version=2026-04-01" \
--resource "https://search.azure.com" \
--headers "Content-Type=application/json" \
--body @/tmp/repro.json
```
**Observed**: HTTP 400
```json
{"error":{"code":"","message":"The API key provided for endpoint 'https://.openai.azure.com/openai/v1/chat/completions' is invalid or has been revoked."}}
```
### Step 3 — Confirm same key still works directly
Repeat the `curl` from Step 1 → HTTP 200, proving the key is valid and the endpoint is reachable from outside Search at the time of the failure.
### Step 4 — Confirm regression is isolated to ChatCompletionSkill
Remove the `ChatCompletionSkill` object from `skills[]` (leaving an empty or non-LLM skillset) and PUT again → HTTP 201/200.
### Environment
- **Azure AI Search**: Standard tier, region `japaneast`, public network enabled, system-assigned + user-assigned MI both enabled
- **Azure AI Foundry resource** (kind `AIServices`): region `japaneast`, `defaultAction: Allow` on `networkAcls`, `disableLocalAuth: null` (local auth enabled), public network enabled
- **Chat deployment**: `gpt-5.2` (GlobalStandard, model version `2025-12-11`)
- **Client**: Azure CLI `az rest` v2+ and Azure Portal skillset editor (both reproduce)
- **Search MI role assignments verified**:
- System MI → `Cognitive Services OpenAI User` at RG scope, `Cognitive Services User` at Foundry resource scope
- UAMI → same (inherited from RG)
- **Search API versions tested**: `2025-11-01-preview`, `2026-04-01`
- **Network**: no private endpoints, direct public access; `curl` reachable to same URL from the same host running `az rest`
Contributor guide
Research direction
Start with the linked specification/search/data-plane/Search/stable/2026-04-01/search.json and review the ChatCompletionSkill schema and validation behavior. Reproduce the skillset PUT with the provided JSON and compare it with the direct curl request and the working embedding-skill case. Done means a well-formed ChatCompletionSkill PUT succeeds with the declared authentication, or the specification and service behavior are aligned.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure
- Domain
- api, cloud
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100