Azure / Azure/azure-rest-api-specs

[BUG] ChatCompletionSkill PUT fails validation on 2026-04-01

Open
#42,311 0 comments 1 reaction 0 assignees View on GitHub
bug customer-reported data-plane question Search Service Attention
Dominant language
TypeSpec
Stars
3.1k
Forks
5.9k
Avg merge
2d 22h
Merged PRs (30d)
444

Description

### API Spec link

https://github.com/Azure/azure-rest-api-specs/blob/main/specification/search/data-plane/Search/stable/2026-04-01/search.json

### API Spec version

2026-04-01

### Describe the bug

Creating or updating an Azure AI Search skillset that contains `#Microsoft.Skills.Custom.ChatCompletionSkill` via the skillset PUT endpoint fails with HTTP 400 and the message:

> `The API key provided for endpoint 'https://.openai.azure.com/openai/v1/chat/completions' is invalid or has been revoked.`

The spec permits the submitted configuration (the body conforms to the `ChatCompletionSkill` schema), and the target Azure OpenAI / AI Foundry endpoint accepts the same `apiKey` when called directly (HTTP 200). The failure is exclusive to the Search skillset validator executed during PUT. The same skillset body, with the `ChatCompletionSkill` removed, is accepted by the service — so the defect is isolated to the eager validation path of `ChatCompletionSkill`.

The bug reproduces across:
- Both REST API versions (`2025-11-01-preview`, `2026-04-01`)
- Both supported URI formats (`/openai/v1/chat/completions`, `/openai/deployments/{name}/chat/completions`)
- All three documented auth mechanisms:
- `apiKey` (fresh key rotated and verified via direct `curl`)
- `authIdentity` with user-assigned MI
- Implicit system-assigned MI (both `apiKey` and `authIdentity` omitted / `null`)
- Setting `authIdentity` to `#Microsoft.Azure.Search.DataNoneIdentity` → returns HTTP 500 instead
- Both REST (`az rest`) and Azure Portal skillset editor

As a practical consequence, once such a skillset is deleted, **it cannot be recreated** via any public API surface, because every well-formed body containing `ChatCompletionSkill` is rejected at validation.

### Expected behavior

Per spec, a PUT request with a body containing a properly shaped `ChatCompletionSkill` and valid `apiKey` (or a correctly configured `authIdentity` / system-assigned MI with required RBAC roles on the target resource) should succeed with HTTP 201/200. Eager validation — if performed — should use the auth material declared in the request body and should accept a valid key that is demonstrably functional at the target endpoint.

Equivalent submissions for `#Microsoft.Skills.Text.AzureOpenAIEmbeddingSkill` on the same skillset against the same AI Foundry resource with the same auth settings are accepted, confirming the declared auth/RBAC posture is correct.

### Actual behavior

PUT returns HTTP 400:

```json
{
"error": {
"code": "",
"message": "The API key provided for endpoint 'https://.openai.azure.com/openai/v1/chat/completions' is invalid or has been revoked."
}
}
```

The same URL and key used out-of-band return HTTP 200:

```
$ curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
-X POST 'https://.openai.azure.com/openai/v1/chat/completions' \
-H 'api-key: ' \
-H 'Content-Type: application/json' \
-d '{"model":"gpt-5.2","messages":[{"role":"user","content":"hi"}]}'
HTTP 200
```

Summary of attempts (all return the identical 400 message, unless noted):

| # | `apiKey` | `authIdentity` | `uri` path | API version | Result |
|---|---|---|---|---|---|
| 1 | omitted | omitted | `/openai/v1/chat/completions` | `2025-11-01-preview` | 400 |
| 2 | `null` | `null` (explicit) | `/openai/v1/chat/completions` | `2025-11-01-preview` | 400 |
| 3 | `null` | `DataNoneIdentity` | `/openai/v1/chat/completions` | `2025-11-01-preview` | **500** |
| 4 | `null` | UAMI | `/openai/v1/chat/completions` | `2025-11-01-preview` | 400 |
| 5 | `null` | UAMI | `/openai/v1/chat/completions` | `2026-04-01` | 400 |
| 6 | valid key (curl HTTP 200 confirmed) | omitted | `/openai/v1/chat/completions` | `2025-11-01-preview` | 400 |
| 7 | valid key | omitted | `/openai/deployments/gpt-5.2/chat/completions` | `2025-11-01-preview` | 400 |
| 8 | valid key | omitted | `/openai/deployments/gpt-5.2/chat/completions` | `2026-04-01` | 400 |

Removing `ChatCompletionSkill` entirely (keeping `ContentUnderstandingSkill` + `AzureOpenAIEmbeddingSkill` with `cognitiveServices` = `#Microsoft.Azure.Search.AIServicesByIdentity` + `identity: null`) → **PUT succeeds**. This isolates the regression to `ChatCompletionSkill`.

### Reproduction Steps

### Prerequisites

- An Azure AI Search service (any tier, Japan East verified)
- An Azure AI Foundry resource (kind `AIServices`) in an adjacent region, with a `gpt-5.2` chat deployment (or substitute any current chat model)
- A valid admin key or AAD token with data-plane access to the Search service

### Step 1 — Verify the target endpoint and key are healthy (out-of-band)

```bash
FOUNDRY_KEY=$(az cognitiveservices account keys list \
--name --resource-group --query key1 -o tsv)

curl -sS -o /dev/null -w "HTTP %{http_code}\n" \
-X POST "https://.openai.azure.com/openai/v1/chat/completions" \
-H "api-key: $FOUNDRY_KEY" \
-H 'Content-Type: application/json' \
-d '{"model":"gpt-5.2","messages":[{"role":"user","content":"hi"}]}'
# Expected: HTTP 200
```

### Step 2 — Attempt skillset PUT with the same key

```bash
cat > /tmp/repro.json <.openai.azure.com/openai/v1/chat/completions",
"apiKey": "$FOUNDRY_KEY",
"inputs": [
{ "name": "systemMessage", "source": "='You are helpful.'" },
{ "name": "userMessage", "source": "='Hi'" }
],
"outputs": [ { "name": "response", "targetName": "out" } ],
"commonModelParameters": { "model": "gpt-5.2" }
}
]
}
EOF

az rest --method put \
--uri "https://.search.windows.net/skillsets/repro-chatcompletion-bug?api-version=2026-04-01" \
--resource "https://search.azure.com" \
--headers "Content-Type=application/json" \
--body @/tmp/repro.json
```

**Observed**: HTTP 400

```json
{"error":{"code":"","message":"The API key provided for endpoint 'https://.openai.azure.com/openai/v1/chat/completions' is invalid or has been revoked."}}
```

### Step 3 — Confirm same key still works directly

Repeat the `curl` from Step 1 → HTTP 200, proving the key is valid and the endpoint is reachable from outside Search at the time of the failure.

### Step 4 — Confirm regression is isolated to ChatCompletionSkill

Remove the `ChatCompletionSkill` object from `skills[]` (leaving an empty or non-LLM skillset) and PUT again → HTTP 201/200.

### Environment

- **Azure AI Search**: Standard tier, region `japaneast`, public network enabled, system-assigned + user-assigned MI both enabled
- **Azure AI Foundry resource** (kind `AIServices`): region `japaneast`, `defaultAction: Allow` on `networkAcls`, `disableLocalAuth: null` (local auth enabled), public network enabled
- **Chat deployment**: `gpt-5.2` (GlobalStandard, model version `2025-12-11`)
- **Client**: Azure CLI `az rest` v2+ and Azure Portal skillset editor (both reproduce)
- **Search MI role assignments verified**:
- System MI → `Cognitive Services OpenAI User` at RG scope, `Cognitive Services User` at Foundry resource scope
- UAMI → same (inherited from RG)
- **Search API versions tested**: `2025-11-01-preview`, `2026-04-01`
- **Network**: no private endpoints, direct public access; `curl` reachable to same URL from the same host running `az rest`

Contributor guide

Open the contributing guide

Research direction

Start with the linked specification/search/data-plane/Search/stable/2026-04-01/search.json and review the ChatCompletionSkill schema and validation behavior. Reproduce the skillset PUT with the provided JSON and compare it with the direct curl request and the working embedding-skill case. Done means a well-formed ChatCompletionSkill PUT succeeds with the declared authentication, or the specification and service behavior are aligned.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure
Domain
api, cloud
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.