P2-B (parent): Long-tail guardrail integrations (~25 vendor guardrails)

Summary

The original plan here was a backlog of ~25 individual vendor guardrail integrations (Pangea, Aporia, Prisma AIRS, CrowdStrike, Zscaler, HiddenLayer, …). Building them one-by-one is the wrong first move. The headline of this issue is now a single platform capability that makes most of that backlog unnecessary:

A generic Webhook (BYO / HTTP) guardrail kind that calls any third-party security service over HTTP, with zero per-vendor data-plane code.

Schedule the bespoke crates only when a customer asks for one that the webhook genuinely cannot reach.

---

1. The universal mechanism: a webhook guardrail kind

One new GuardrailKind::Webhook(WebhookConfig). The data plane POSTs the in-flight request (and, on the output hook, the response) plus metadata to a customer/partner-configured URL, and acts on the JSON reply. Because almost every guardrail vendor in the backlog exposes an HTTP API, this single integration covers the long tail without a new crate per vendor.

Request the DP sends (to the configured url)

POST {url}
Content-Type: application/json
X-AISIX-Signature: {hmac_sha256(body, signing_secret)}   // optional, lets the receiver verify it came from us
{
  "hook_point": "input" | "output",
  "request":  { /* OpenAI-shaped chat request, current in-flight payload */ },
  "response": { /* OpenAI-shaped chat response, present only on the output hook */ },
  "metadata": { "model": "...", "guardrail_id": "...", "request_id": "..." }
}

Header policy (security — default-deny). The DP does not forward the caller's inbound headers to the webhook: the client Authorization, cookies, and API keys are never propagated. If the webhook itself needs auth, that's a separate operator-configured credential_id sent only to the webhook URL — never the client's secret. An optional forward_headers allow-list may echo a few non-sensitive headers; values outside the list are redacted to "[present]" rather than sent. (Both reference gateways do this; Portkey strips request headers entirely — Portkey-AI/gateway webhook.ts:67-74.)

Reply the DP acts on

{
  "verdict": "allow" | "block" | "flag",
  "reason": "string, operator-facing only (never sent to the caller)",
  "transformedData": { "request": { /* ... */ }, "response": { /* ... */ } }  // optional, direction-gated
}

Transform is direction-gated: transformedData.request is honored only on the input hook; transformedData.response only on the output hook. The input direction works today; the output direction is inert until output-rewrite lands (§2.2). This mirrors Portkey's gating (webhook.ts:83-96).

Verdict mapping (reuses the existing GuardrailVerdict)

Webhook reply	DP verdict	Notes
allow	Allow	passthrough
block	Block { reason }	422; reason kept for logs, never leaked to the client wire envelope (per #153)
flag	Allow + telemetry flag	"observe" outcome; needs the monitor-mode plumbing in §2.4
allow + transformedData.request (input hook)	Rewrite { payload }	works today — see §2.1
allow + transformedData.response (output hook)	coerced to Allow for now	blocked on output-rewrite — see §2.2
unreachable (timeout / 5xx / network)	Bypass if fail_open else Block	only transport failures count as "unreachable"; a well-formed block reply always blocks

Config fields (CP + UI)

url, optional credential_id for the webhook's own auth (org-scoped encrypted secret, reused from the Bedrock/credential mechanism in #2625 — never the client's key), hook_point (input/output/both), timeout_ms (explicit; e.g. default 3000 ms), fail_open, mode (enforce/monitor, see §2.4), optional forward_headers allow-list (values redacted to "[present]" outside the list), optional HMAC signing_secret.

Competitive reference (Portkey BYO webhook): https://portkey.ai/docs/integrations/guardrails/bring-your-own-guardrails — same shape (POST request/response, act on a verdict, optional transform). Confirm our exact field names against our own OpenAI-shaped ChatFormat/ChatResponse, not Portkey's.

---

2. Platform primitives the webhook (and the rest of the catalog) need

Verified against origin/main (2026-05-29). These unlock the whole catalog, not just the webhook — recommend extracting them as their own P1 issues (sequencing in §6).

2.1 Input-path transform — ✅ already works

GuardrailVerdict::Rewrite { payload } is defined (crates/aisix-guardrails/src/lib.rs:69), propagated through the chain via Cow<ChatFormat> (crates/aisix-guardrails/src/chain.rs check_input, substitutes the payload for downstream guardrails), and substituted before upstream dispatch in crates/aisix-proxy/src/chat.rs:691. So a webhook returning transformedData.request on the input hook (redaction, normalization) ships with no platform prerequisite. No shipped guardrail emits Rewrite yet — the webhook (or Presidio in #52) is the first producer.

2.2 Output-path response rewrite — ❌ the highest-value gap

chain.rs check_output takes an immutable &ChatResponse and ignores Rewrite ("output rewrites would need a mutable resp which the trait doesn't provide. Treat as Allow."); chat.rs:1357 and the streaming path at chat.rs:2266 do the same. Today's GuardrailVerdict::Rewrite only carries Box<ChatFormat> (a request), so it cannot express a rewritten response.

Concrete change:

• Add a response-carrying outcome — e.g. a RewriteOutput(Box<ChatResponse>) variant, or generalize Rewrite's payload to an enum over request/response — and thread an owned response through GuardrailChain::check_output, mirroring the allocation-free Cow<ChatFormat> pattern already used on the input path.
• crates/aisix-proxy/src/chat.rs then substitutes the rewritten response before serializing to the client.
• Non-streaming first; defer streaming-body rewrite to #223. This isn't a shortcut: two mature open-source gateways implement output response-transform for non-streaming 200s and both deliberately punt on streaming-body rewrite (Portkey applies the transform at responseHandlers.ts:287-289 and skips the stream body at :262-277). That validates the scope split.

This one change unblocks Presidio output-redaction (#52), Bedrock mask-feedback (#51), and the webhook's output-hook transformedData.response.

Backward-compatible by construction. Prefer the additive RewriteOutput(Box<ChatResponse>) variant over changing Rewrite's existing payload, and keep the trait signature check_output(&self, resp: &ChatResponse) -> GuardrailVerdict unchanged. The shipped Bedrock and Azure-CS impls only construct verdicts (Allow / Block / Bypass) and never exhaustively match on GuardrailVerdict in production code — so they need zero changes. Only GuardrailChain::check_output, crates/aisix-proxy/src/chat.rs, and the custom PartialEq on the enum gain a new match arm.

2.3 Shared content extract/inject helper — the DRY foundation

Every text guardrail (keyword, Lakera, Presidio, OpenAI Moderation, webhook) otherwise re-walks ChatFormat messages itself. Add one helper in aisix-guardrails — content::extract(&ChatFormat) -> Vec<&str> and content::replace(ChatFormat, Vec<String>) -> ChatFormat (plus the ChatResponse equivalents once §2.2 lands) — so each guardrail maps over normalized text instead of hand-walking the schema. Both reference gateways centralize this (Portkey getCurrentContentPart/setCurrentContentPart, utils.ts:59-209); it keeps each vendor integration ~100-200 lines and makes the §2.2 rewrite uniform.

2.4 Observe / monitor mode — wire the already-shipped enforcement_mode field

The config field already exists: Guardrail.enforcement_mode (crates/aisix-core/src/models/guardrail.rs, values "block" default / "monitor"), but its own doc-comment says "not yet implemented; the DP currently always blocks regardless of this field." So this is wiring, not a new field: the chain post-processes — when an enforcement_mode="monitor" guardrail returns Block, record a new guardrail_flagged_reason telemetry field (parallel to the existing usage_events.guardrail_bypassed_reason) and continue. Keep guardrail impls mode-agnostic; the default "block" preserves today's behavior for the shipped Bedrock/Azure-CS rules. Both reference gateways treat log-only as table stakes (Portkey's non-deny path tags the call and attaches results: index.ts:265-268). Portkey product: https://portkey.ai/docs/product/guardrails

2.5 Concurrent execution at a hook point — drop the all-sequential chain

The chain runs guardrails strictly sequentially today, so N remote guardrails cost N× latency. Run independent guardrails concurrently at a hook point, falling back to sequential only when a rewrite-producer is in the chain (rewrites must thread in order). Reference: Portkey runs a hook's checks via Promise.all unless sequential is set (index.ts:366-417).

Precedence must be preserved — this is a latency optimization, never a semantics change. The chain resolves guardrails in priority/scope order and the first Block wins via an early return (crates/aisix-guardrails/src/chain.rs check_input). Concurrency must keep that deterministic precedence: dispatch in parallel, then fold the results in the original resolved order so the same Block wins and Bypass-shadowing is identical. The shipped Bedrock/Azure-CS rules must enforce exactly as they do today, only faster.

2.6 Per-model / per-key scoping + ordering — ✅ mostly shipped; only direction filtering is the gap

Per-request scoping is already shipped and wired, not new work: GuardrailScopeType {Env, Model, ApiKey, Team} + GuardrailAttachment {guardrail_id, scope_type, scope_id, priority, enabled} (crates/aisix-core/src/models/guardrail.rs), resolved per request by GuardrailIndex::resolve(RequestContext { model_id, api_key_id, team_id }) — called at crates/aisix-proxy/src/chat.rs:654-660, deduping by guardrail_id and ordering by priority desc then scope specificity ApiKey > Team > Model > Env (crates/aisix-guardrails/src/index.rs). This already satisfies the security requirement: scope and opt-out are resolved entirely from server-side attachment config keyed on the authenticated api_key_id / team_id, never from the client request body, so a caller cannot disable their own guardrails. The one remaining gap is direction filtering: the Guardrail.direction field (default "both") exists but resolve does not yet filter attachments by it — that is the only piece of this section still to wire, and it is already server-side.

2.7 fail-open default + the already-shipped mandatory field

Keep the per-guardrail fail_open flag (correct; crates/aisix-core/src/models/guardrail.rs, default true), but document the default direction by category: availability-sensitive checks may fail-open; injection / PII blockers should default fail-closed (if the check can't run, block). The two reference gateways diverge here, which is exactly why this belongs to per-guardrail config rather than one global default. A related field already exists but is inert: Guardrail.mandatory (default false) is documented as "not yet implemented … fail_open alone governs." Wiring fail-closed therefore means honoring mandatory=true as a hard override (block on bypass regardless of fail_open), not adding a new field — injection / PII blockers set mandatory=true.

---

3. What the webhook does not cover (the residual bespoke list)

Keep bespoke crates only for services the webhook cannot reach: non-HTTP / SDK-only transports, unusual auth handshakes, or self-hosted components that need in-process calls. When one is scheduled by customer ask, spawn a child issue using the template in §5.

Verified against Portkey's partner catalog (https://portkey.ai/docs/product/guardrails/list-of-guardrail-checks) — most of these are plain HTTP and therefore reachable via the webhook, listed here only so we can confirm a customer ask before writing any code:

Backlog item	Category	Webhook-reachable?	Reference
Pangea AI Guard	injection + PII	yes (HTTP)	/integrations/guardrails/pangea
Aporia	custom policy	yes (HTTP)	/integrations/guardrails/aporia
Pillar / Lasso / Prompt Security / Qualifire / Javelin / Akto	injection / PII / toxicity	yes (HTTP)	listed in catalog
Palo Alto Prisma AIRS, CrowdStrike AIDR, Zscaler	enterprise AI guards	yes (HTTP)	listed in catalog
Patronus	output-quality / toxicity / PII evals	yes (HTTP)	/integrations/guardrails/patronus-ai
Google Cloud Model Armor	moderation	confirm vs primary docs	not in Portkey catalog
IBM Granite Guardian	moderation	confirm vs primary docs	not in Portkey catalog
Meta Llama-Guard / PromptGuard	self-hosted classifier	maybe (HTTP if served)	not in Portkey catalog
AIM / HiddenLayer / DynamoAI / EnkryptAI / GraySwan / Noma / Onyx	misc security	confirm vs primary docs	not in Portkey catalog
LLM-as-a-Judge	configurable LLM-graded check	likely an internal model call, not a webhook	—

For any item not in Portkey's catalog, confirm the vendor's official API against primary docs when its child issue is scheduled; do not assume a request/response shape.

---

4. Already shipped (so it's not re-scoped here)

• Azure Content Safety — Prompt Shield slice shipped: PR #423 (merged 2026-05-27) + schema fix #439 (merged 2026-05-29). Covers jailbreak / prompt-injection detection via Azure's /contentsafety/text:shieldPrompt. The content-moderation slice (hate/violence/sexual/self-harm via Azure's text:analyze) is not done — open a child issue only if a customer asks.
• Bedrock Guardrails — see #51. Lakera / Presidio / OpenAI Moderation — see #52.

---

5. Child-issue template (for a bespoke vendor, when scheduled)

Every child spawned from the residual list must fill in:

• Background — who asks for it, which category (injection / PII / moderation / topic / quality), and why this vendor needs bespoke code instead of the webhook.
• Research — the vendor's official API (endpoint, auth, request/response) cited to primary docs; input-only / output-only / both; whether it needs redact/transform.
• UI design — the new kind entry + form fields (keys as secrets, endpoints, categories/thresholds, action, hook_point), following the existing guardrails-page conventions.
• Implementation — Guardrail trait impl (built on the §2.3 content helper), new GuardrailKind variant + schema oneOf branch + CP kind; verdict mapping; fail-open behavior.
• Testing & acceptance — mock the external service in mock-llm; real-chain e2e for block / redact / flag as applicable; schema accept/reject; Dashboard CRUD Playwright.
• Docs — docs/configuration/guardrails.md section + a short tutorial.

6. Suggested ordering

1. Webhook guardrail kind (§1) + input-redact (§2.1) + content helper (§2.3) — unlocks the long tail.
2. Guardrail actions — output response-rewrite (§2.2) + monitor mode (§2.4).
3. Chain execution — concurrency (§2.5) + per-model/per-key scoping & ordering (§2.6).
4. Bespoke vendors (§3) by customer ask, preferring webhook-reachable ones (zero net-new DP code) before writing a crate.