Azure / Azure/azure-dev

azure.ai.agents: schema for `Policy` is looser than azd's own validation

Open
#9,653 0 comments 0 reactions 0 assignees View on GitHub
area/schema bug ext-agents
Dominant language
Go
Stars
569
Forks
364
Avg merge
2d 19h
Merged PRs (30d)
136

Description

### Context

Follow-up from review of #9596 ([discussion](https://github.com/Azure/azure-dev/pull/9596#discussion_r3789411790)). That PR added `invocationsModeration` to `rai_policy` policies and encoded its *internal* conditional-required rules in `schemas/azure.ai.agent.json`. It deliberately did **not** tighten the surrounding `Policy` schema, which has a pre-existing class of gaps where the published schema is looser than what `azd` actually accepts.

The effect is that an editor green-lights configuration that fails at `azd package`/`deploy` time — the schema stops being a reliable authoring aid.

### Gaps

1. **`Policy.type` is a bare `{"type": "string"}`** with no `enum`. The schema accepts any policy type, while Go rejects everything but `rai_policy` in its `default:` branch.

2. **`raiPolicyName` is not required and has no `minLength`.** `{"type": "rai_policy"}` with no name, or with `""`, validates against the schema but is rejected by `azd`. This wants an `if`/`then` keyed on `type`.

3. **No root conditional gating `invocationsModeration` on agent shape.** The schema accepts the block when `kind` is not `hosted`, or when `protocols` is omitted or contains only `responses`/`invocations_ws` — all of which `azd` rejects. This is the specific item [Copilot flagged](https://github.com/Azure/azure-dev/pull/9596#discussion_r3788611857): when any policy carries `invocationsModeration`, require `kind: hosted` **and** a `protocols` entry with `protocol: invocations`.

4. **More than one `rai_policy` entry.** #9596 made this an error in Go (`rai_config` is singular on the wire, so only the first would ever reach the service). The schema still permits a list of them.

### Why it is separate

These are all *pre-existing* schema/Go divergences on `Policy` — items 1, 2 and 4 are not specific to `invocationsModeration`. Fixing them together in one pass keeps the conditional logic coherent, and item 3's root-level conditional is easier to write once the `type` enum from item 1 exists. Folding them into #9596 would have mixed a schema-hardening concern into a feature PR.

### Notes

- The schema file is CRLF; edit it as text rather than round-tripping through a JSON serializer, which would rewrite every line.
- `Policy` is `additionalProperties: false`, so schema and Go must move together.
- Worth adding a test that asserts schema-accepts ⇔ Go-accepts for the `Policy` shape, so this class of drift is caught rather than re-discovered in review.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.