Does a usage-based billing spending policy alone unblock the A2A 403 "NoPolicy: AI credits access is not configured for this user"?

Question

Before asking our tenant admins to enable usage-based billing, we'd like to confirm what exactly the Agent Evaluations CLI requires, since the billing prerequisite doesn't appear in the CLI docs (overview / troubleshooting) and the usage-based billing docs currently list only Copilot Cowork and Work IQ API as covered services.

Is activating a Copilot Credits spending policy (M365 admin center → Copilot → Cost Management) that scopes the eval-running user sufficient to unblock runevals, or are there additional requirements (e.g. an M365 Copilot license for the user, specific service enablement for the A2A path, propagation delays)?

What we observed

• CLI version: 1.11.0, Windows 11, Node 22
• Agent: declarative agent (Agents Toolkit project), provisioned to dev, works normally in the m365.cloud.microsoft/chat UI for the same user
• Interactive M365 auth succeeds, agent ID resolves from M365_TITLE_ID
• Every prompt send fails with:

Prompt "..." failed after 1 attempt: A2A request failed (HTTP 403 Forbidden).
Body: {"error":"NoPolicy","message":"AI credits access is not configured for this user. Please request access from your admin."}

• Note: the run still exits 0 and prints "Evals completed successfully!" — item-level failures only surface in the JSON results. A related question: is a non-zero exit (or a flag) planned for runs where all items errored? That would help CI usage.

What would help

1. Confirmation of the exact tenant prerequisites for the A2A path used by runevals (spending policy? license? both?)
2. Whether the eval traffic appears under a named service in the Cost Management "Agents and services" view, so admins can scope a policy narrowly to it
3. Rough credit consumption per eval prompt, so we can size a capped policy for a dev security group
4. A docs update adding these prerequisites to the CLI overview/troubleshooting pages

Thanks — the tool itself worked flawlessly right up to the policy gate, and we'd love to adopt it for pre-provision regression testing of our agents.

Start with the CLI overview and troubleshooting pages, which the issue identifies as missing the billing prerequisite. Confirm the tenant requirements for the A2A path, service naming and credit guidance, then document them and clarify whether item-level failures should produce a non-zero exit or flag.