EPIC: Guardrails and Fact Reasoner

Goal

Bring the two guardrail services — Risk Agent and Granite Guardian — into AKD Labs and establish that they are trustworthy enough to run on every agent turn.

What this epic set out to do

1. Establish whether the guards actually work
They were already wired into agent chat, but nobody had probed them adversarially. This was to stress-test both against real agent output and write down every failure mode, so we would know what we were shipping rather than trusting a green verdict.

What that surfaced: the output guard was passing the user's prompt as the context claims are judged against, which inverted the check — an answer that agreed with a false premise passed, one that corrected the user was flagged. Alongside it: nondeterministic verdicts on identical input, dilution on long answers, hedging rescuing real violations, verdict/rationale incoherence, and Granite Guardian false-positiving on full artifact text.

2. Let teams define their own risks
The Risk Agent shipped with a fixed set of five scientific-integrity risks. This was to let an agent owner supply their own, since a governance agent and a science agent do not share a risk model. Scope was a toggle plus either a form (id / description / criteria) or pasted YAML, stored with the agent's existing config.

The important discovery was semantic, not technical: supplying custom risks REPLACES the service's defaults rather than adding to them, so enabling them silently turns off hallucination and attribution checking unless the user re-declares those. The UI had to say so plainly.

Added beyond the original scope

FactReasoner was not part of the original intent — the epic was Risk Agent and Granite Guardian. It came in as an opt-in factuality check a user can run on demand from chat, which in turn required work on the service deployment it runs against.

Definition of done

• Both guards' failure modes documented, and the correctness bugs fixed where they are ours or filed upstream where they are not.
• Custom risks configurable per agent, with the replacement behaviour visible to whoever turns them on.
• FactReasoner reachable from chat on demand, as an extra rather than a guardrail.