microsoft / microsoft/agent-governance-toolkit

BackendRegistry / OPAPolicyBackend / CedarPolicyBackend are never consulted by govern() - registering a backend is a silent no-op

Open
#3,911 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.3k
Forks
1.1k
Avg merge
5d 11h
Merged PRs (30d)
142

Description

**What's misleading**

`agentmesh/governance/backend.py` documents `BackendRegistry` as the way to plug in external policy engines (OPA, Cedar, custom):

```python
# Register a backend
BackendRegistry.register(my_opa_evaluator)

# Retrieve by name
backend = BackendRegistry.get("opa")
decision = backend.evaluate("export", {"agent": {"role": "analyst"}})
```

and `OPAPolicyBackend`/`CedarPolicyBackend` both exist and correctly implement the `ExternalPolicyBackend` protocol. Reasonable to assume that once registered, `govern()`'s own policy evaluation would use a registered backend - the module's own docstring says backends are "used interchangeably by the governance layer."

**What actually happens**

`govern()`'s enforcement path (`govern.py`'s `GovernedCallable.__call__` → `PolicyEngine.evaluate` in `policy_evaluator.py`... actually via `self._engine.evaluate(self._config.agent_id, context)`) never references `agentmesh.governance.backend` or `BackendRegistry` at all - confirmed with `grep -rl BackendRegistry` across the whole `governance/` package: only `backend.py`, `opa.py`, `cedar.py`, and `__init__.py` (the re-export) reference it. Registering a backend with `BackendRegistry.register(...)` has no effect on what `govern()` does; its `Policy`/`PolicyRule` YAML/JSON engine is the only thing that runs, always.

**Impact**

Anyone reaching for OPA/Cedar because the built-in YAML condition DSL can't express what they need (see also: it's regex-based and doesn't support field-vs-field comparisons - happy to file that separately if useful) will reasonably try `BackendRegistry.register()` first, based on the docstring, and get silent no-op behavior - no error, the governed call just keeps using the YAML engine's `default_action`.

**Suggested fix / ask**

Either:
- Have `govern()`/`PolicyEngine` actually consult `BackendRegistry` when a policy rule specifies an external backend (e.g. a `backend: opa` field on `PolicyRule`), or
- Update `backend.py`'s docstring and the OPA/Cedar module docs to make clear this is a standalone, manually-wired API - not something `govern()` picks up automatically.

Happy to share the minimal repro (a two-file example showing `BackendRegistry.register()` having zero effect on a `govern()`-wrapped call) if useful.

Contributor guide

Open the contributing guide

Research direction

Start with govern.py's GovernedCallable.__call__ and policy_evaluator.py's PolicyEngine.evaluate, then compare that path with BackendRegistry in backend.py and the integrations in opa.py and cedar.py. Determine whether the intended behavior is explicit backend selection or documentation-only standalone use; done means registered backends no longer silently mislead users and the relevant behavior or documentation is covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.