microsoft / microsoft/agent-governance-toolkit

Add an async evaluation path to PolicyEngine so external backends aren't forced into sync-over-async

Open
#3,775 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
6.3k
Forks
1.1k
Avg merge
5d 11h
Merged PRs (30d)
142

Description

## Summary

`IExternalPolicyBackend` exposes an async `EvaluateAsync`, but `PolicyEngine` only ever calls the synchronous `Evaluate`. Any external backend whose work is inherently async (a network call to a remote policy service) is forced to block a thread with `.GetAwaiter().GetResult()`. Could `PolicyEngine` (and the kernel/MCP paths above it) offer an async evaluation path that awaits `IExternalPolicyBackend.EvaluateAsync`?

## Current behavior (v5.0.0)

- `IExternalPolicyBackend` defines both `Evaluate(context)` and `EvaluateAsync(context, ct)` (the latter defaulting to `Task.FromResult(Evaluate(context))`).
- `PolicyEngine.Evaluate(string agentDid, Dictionary context)` is synchronous and invokes external backends synchronously, roughly:

```csharp
var externalDecisions = _externalBackends
.Select(backend => backend.Evaluate(evalContext)) // sync only
.ToList();
```

- There is no `PolicyEngine.EvaluateAsync`, so `EvaluateAsync` on a backend is never called by the engine, and the kernel entry points (`EvaluateToolCall`) are synchronous as well.

*(Line numbers approximate for 5.0.0; happy to point at exact locations.)*

## Why it's a problem

A backend that calls a remote service to make its decision has no truly synchronous option. In our case we're building an external backend over Amazon Bedrock Guardrails, whose SDK is async-only. To satisfy the sync `Evaluate`, we must bridge:

```csharp
public ExternalPolicyDecision Evaluate(IReadOnlyDictionary context)
=> EvaluateAsync(context).GetAwaiter().GetResult();
```

This blocks a thread for the full round-trip on every tool call. Under load that pressures the thread pool and can cause starvation. The in-box OPA backend avoids this only because `HttpClient` still exposes a synchronous `Send`; SDK-based backends generally don't have that escape hatch.

Notably the MCP tool path (`GovernedMcpServerTool.InvokeAsync`) is already `async`, so there's an async context available that could `await` an async engine, but it currently calls into the synchronous engine.

## Proposed solution

Add an async evaluation path that awaits backends' `EvaluateAsync`:

```csharp
public Task EvaluateAsync(
string agentDid,
Dictionary context,
CancellationToken cancellationToken = default);
```

- Evaluate external backends with `await backend.EvaluateAsync(evalContext, ct)`
- Optionally thread an async path through `GovernanceKernel.EvaluateToolCall` / the MCP decorator so callers already in an async context never block.
- Keep the synchronous `Evaluate` for backward compatibility (it can delegate or remain as-is).

## Alternatives considered

- **Sync-over-async in the backend** (current workaround): works but blocks a thread per call and risks pool starvation.
- **Sync SDK path**: not available for async-only SDK clients.

## Willing to contribute

Happy to open a PR if the maintainers are open to the direction.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing PolicyEngine.Evaluate and IExternalPolicyBackend.EvaluateAsync, then follow GovernanceKernel.EvaluateToolCall and GovernedMcpServerTool.InvokeAsync. Add an async evaluation path that propagates cancellation and awaits external backends, while preserving the synchronous API. Done means async MCP and kernel callers avoid blocking and existing synchronous callers remain supported.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend-api-design, security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.