Azure / Azure/azure-dev

azure.ai.agents: schema for `Policy` is looser than azd's own validation

Open
#9,653 0 comments 0 reactions 0 assignees View on GitHub
area/schema bug ext-agents
Dominant language
Go
Stars
569
Forks
364
Avg merge
2d 22h
Merged PRs (30d)
139

Description

### Context

Follow-up from review of #9596 ([discussion](https://github.com/Azure/azure-dev/pull/9596#discussion_r3789411790)). That PR added `invocationsModeration` to `rai_policy` policies and encoded its *internal* conditional-required rules in `schemas/azure.ai.agent.json`. It deliberately did **not** tighten the surrounding `Policy` schema, which has a pre-existing class of gaps where the published schema is looser than what `azd` actually accepts.

The effect is that an editor green-lights configuration that fails at `azd package`/`deploy` time — the schema stops being a reliable authoring aid.

### Gaps

1. **`Policy.type` is a bare `{"type": "string"}`** with no `enum`. The schema accepts any policy type, while Go rejects everything but `rai_policy` in its `default:` branch.

2. **`raiPolicyName` is not required and has no `minLength`.** `{"type": "rai_policy"}` with no name, or with `""`, validates against the schema but is rejected by `azd`. This wants an `if`/`then` keyed on `type`.

3. **No root conditional gating `invocationsModeration` on agent shape.** The schema accepts the block when `kind` is not `hosted`, or when `protocols` is omitted or contains only `responses`/`invocations_ws` — all of which `azd` rejects. This is the specific item [Copilot flagged](https://github.com/Azure/azure-dev/pull/9596#discussion_r3788611857): when any policy carries `invocationsModeration`, require `kind: hosted` **and** a `protocols` entry with `protocol: invocations`.

4. **More than one `rai_policy` entry.** #9596 made this an error in Go (`rai_config` is singular on the wire, so only the first would ever reach the service). The schema still permits a list of them.

### Why it is separate

These are all *pre-existing* schema/Go divergences on `Policy` — items 1, 2 and 4 are not specific to `invocationsModeration`. Fixing them together in one pass keeps the conditional logic coherent, and item 3's root-level conditional is easier to write once the `type` enum from item 1 exists. Folding them into #9596 would have mixed a schema-hardening concern into a feature PR.

### Notes

- The schema file is CRLF; edit it as text rather than round-tripping through a JSON serializer, which would rewrite every line.
- `Policy` is `additionalProperties: false`, so schema and Go must move together.
- Worth adding a test that asserts schema-accepts ⇔ Go-accepts for the `Policy` shape, so this class of drift is caught rather than re-discovered in review.

Contributor guide

Open the contributing guide

Research direction

Start with schemas/azure.ai.agent.json, preserving its CRLF line endings, then locate the Go validation for Policy and rai_config to compare accepted shapes. Add coverage for the four listed schema/Go divergences, including the invocationsModeration agent conditions, and verify that schema acceptance matches Go acceptance.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, go
Domain
cli, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.