[Feat] Threat model for agent manipulation (prompt injection) in a shared deployment

Part of #777. **After 3.0.** Filed to apply AI-specific security guidance to CAO's own content and action paths.

## In plain terms

This issue concerns authorized agents being induced to perform unintended actions. It complements the identity, resource-access and execution-isolation work rather than assuming those tickets only address malicious people.

A CAO agent reads things: files, command output, web pages, messages from other agents, and the contents of a repository. Any of that can contain text written to be read by a language model rather than a human — instructions dressed up as content. If an agent follows them, it does something its user never asked for.

The agent may already be authenticated and still act against its user's intent. In the planned 3.0 system it has restricted delegated CAO authority (#774/#745), not automatically every permission held by its human owner. Assess that actual authority together with the filesystem, network and provider credentials available inside the runtime.

## Why this needs its own issue

General tenancy and container-isolation guidance does not by itself describe CAO's agent-manipulation risks. But the earlier claim that mainstream cloud/security sources do not cover this was too broad. Microsoft's [Prompt Shields documentation](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection) explicitly describes third-party document attacks that lead to unintended commands, and AWS documents [prompt-injection detection and its limits](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-prompt-attack.html).

These are references, not proof that a hosted filter protects every CAO path. For example, the cited Bedrock documentation explicitly excludes tool results from its prompt-attack filter. Provider CLI file access, tool output, memory and inter-agent messages therefore need their own coverage assessment. Also work through:

- **OWASP Top 10 for Large Language Model Applications**
- **NIST AI 100-2**, the adversarial machine learning taxonomy

Applying these sources to CAO and recording what they do and do not cover is the first task here.

## Why it gets worse with multiple tenants

On a laptop, a manipulated agent can misuse reachable local and external resources, including repositories or cloud services its credentials allow. Being local does not limit all effects to that laptop.

In a shared deployment, poisoned repository content, retrieved documents or agent messages can drive unwanted actions within the authority actually granted. Correct authentication and ownership checks prevent unauthorized access; they do not prove that every otherwise-permitted action matches the user's task.

Memory and inter-agent messaging already move content between an owner's agent contexts. Cross-owner user-resource sharing is disabled in 3.0 by #780, so do not assume it already provides a path between ordinary members. Assess reachable paths now, then extend coverage when deliberate sharing is introduced. Untrusted content can originate outside CAO regardless of the number of people or tenants.

## What this would involve

Scoping is part of the work, but the likely shape:

- **A written threat model** for agent manipulation in a multi-tenant CAO, worked from the OWASP LLM Top 10 and NIST AI 100-2.
- **Treating retrieved content as data, not instructions** — the boundary between what an agent is told to do and what it merely reads.
- **Limiting blast radius**, assessing and strengthening the delegated credentials already required by #774/#745, revocation in #779, and runtime boundaries in #784. Include #783's independent service-account actors when that feature lands; do not defer the 3.0 delegation controls to it.
- **Making agent actions reviewable after the fact**, which depends on #782.
- **An honest statement of residual risk.** This class of problem is not solved in the industry, and promising otherwise would be dishonest.

## Acceptance criteria

- [ ] A threat model exists, grounded in the OWASP LLM Top 10 and NIST AI 100-2.
- [ ] Content-carrying paths — memory, inter-agent messages, retrieved documents, tool output — are assessed against it.
- [ ] The model records actual CAO, OS, network and provider authority rather than assuming full human permissions or complete protection from a scoped CAO token.
- [ ] Cases include an owner's own agent chain reading third-party content, as well as the cross-owner paths enabled by later sharing; tenant count alone does not determine exposure.
- [ ] Mitigations are proposed with their limits stated.
- [ ] Residual risk is documented for operators rather than left implied.

## Evidence

| Claim | Where |
| --- | --- |
| Cloud vendors document prompt injection and indirect document attacks | [Microsoft Prompt Shields](https://learn.microsoft.com/en-us/azure/ai-services/content-safety/concepts/jailbreak-detection); [Amazon Bedrock prompt-attack guidance](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-prompt-attack.html) |
| Appropriate sources | OWASP Top 10 for LLM Applications; NIST AI 100-2 |
| Agents exchange content directly | `send_message`, `update_metadata` — *"Metadata you publish here is visible to any sibling that can discover you -- treat it as you would any other inter-agent message, not as private state"* (`mcp_server/server.py:1194-1196`) — and CAO memory |

Start with the OWASP Top 10 for LLM Applications and NIST AI 100-2, then inspect the cited send_message and update_metadata paths in mcp_server/server.py:1194-1196 and the CAO memory paths. Document the threat model, assess content-carrying paths and actual CAO, OS, network, and provider authority, propose bounded mitigations, and record residual risk.

Written by the indexing model from the issue text.