CopilotKit / CopilotKit/OpenBot

[Feature] Content-level governance boundary — PII detection, prompt injection defense, and cost budgets for Bot tool arguments

Open
#86 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
5.2k
Forks
655
Avg merge
11h 56m
Merged PRs (30d)
370

Description

Problem

OpenBot's gateway handles infrastructure-level governance brilliantly — CEL policies on browser URLs, file paths, shell commands, and MCP tool classification. But there's a gap at the content level:

  1. PII in tool arguments — A Bot can pass an SSN, credit card number, or email to a tool without the gateway noticing, because CEL evaluates tool.name and page.url, not the argument content itself.

  2. Prompt injection via tool inputs — If a retrieved document or user message contains "ignore previous instructions...", that flows through the AG-UI endpoint into the Bot's tool arguments. The gateway sees a valid tool call to a permitted target — the adversarial content is invisible to CEL.

  3. Per-session cost budgets — The gateway records actions but doesn't enforce cumulative spend limits. A Bot can loop tool calls indefinitely without hitting a cost ceiling.

  4. Secret leakage in arguments — A Bot might pass api_key=sk-abc123... as a tool argument. The gateway correctly redacts secrets from the audit trail, but doesn't prevent the call from executing with the secret in-flight.

Proposed Solution

Add a content-level governance evaluator that runs alongside CEL, inspecting the content of tool arguments (not just tool names and targets). This would fire between "policy evaluated" and "call the computer" in the gateway pipeline.

One implementation: TealTiger — deterministic, in-process, no LLM in the evaluation path, <2ms per check. Already provides:

Check What It Catches
PII detection SSN, credit card, email, phone in tool args
Secret detection API keys, tokens, credentials
Prompt injection Instruction override, DAN, delimiter injection, system prompt override (8 patterns)
Cost budget Cumulative session spend with hard-stop
Argument validation SQL injection, path traversal in tool inputs

It could integrate as:

Option A — Gateway plugin:

// In the server gateway, after CEL evaluation, before calling computer
const contentDecision = await tealtigerEvaluate({
  toolName: resolvedTarget.tool,
  arguments: resolvedTarget.input,
  botId: bot.id,
  sessionCost: auditRow.cumulativeCost,
});
if (contentDecision.action === "DENY") {
  return refuse(contentDecision.reason);
}

Option B — Boundary preset:
A preset at /admin/boundaries that enables content-level checks:

# Content governance preset
name: "Content Protection"
rules:
  - scope: tool_arguments
    check: pii
    action: deny
    categories: [ssn, credit_card, email]
  - scope: tool_arguments
    check: prompt_injection
    action: deny
  - scope: session
    check: cost_budget
    action: deny
    max_usd: 5.00

Option C — Agent-level middleware:
The AG-UI agent (LangGraph, etc.) wraps itself with TealTiger middleware before registering as a Bot endpoint. Content governance happens inside the agent, infrastructure governance happens at the gateway. Defense-in-depth.

Use Cases
  • Financial services Bot handling account data — must not leak PII in tool arguments to third-party MCP servers
  • Research Bot consuming external documents — needs injection defense before those documents influence tool calls
  • Multi-Bot deployment with cost allocation — each Bot needs a per-session budget that hard-stops regardless of CEL rules
  • Compliance audit — need evidence that content-level checks were evaluated (not just that the URL/tool was permitted)
Why not just extend CEL?

CEL is great for structural checks (tool.name == "browser_navigate" && page.host != "internal.company.com"). But content scanning requires:

  • Regex pattern matching across variable-length text (PII/injection)
  • Stateful tracking (cumulative cost)
  • Confidence thresholds (injection detection isn't binary)

These are better served by a purpose-built content evaluator running alongside CEL rather than shoehorning them into CEL expressions.

Additional Context
  • TealTiger: https://github.com/agentguard-ai/tealtiger (Apache 2.0)
  • Already integrated with AG2 (built-in), Haystack, LangChain, CrewAI, Strands
  • NVIDIA Inception member
  • No external server required — runs in-process
  • Happy to contribute a PR once the boundary plugin interface stabilizes

This is complementary to existing governance, not a replacement. CEL handles infrastructure boundaries. Content evaluation handles what's inside the calls that pass those boundaries.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the server gateway pipeline described in the issue, especially the point after CEL evaluation and before the computer call, and review the existing boundary plugin interface. Compare the proposed gateway plugin, boundary preset, and agent middleware entry points; the work is done when a decided integration path and its content-governance behavior are defined and covered by appropriate project tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend-api-design, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.