awslabs / awslabs/cli-agent-orchestrator

[Feat] Install-time integrity and trust verification for profiles

Open
#682 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.3k
Forks
267
Avg merge
1d 23h
Merged PRs (30d)
70

Description

## Overview

Add a layered trust gate to the profile installation path so that every remote install records provenance, screens content for supply-chain attack patterns, and enforces a configurable trust policy.

Today the install path validates the profile *name* (strict stem validation added in #226) and the *provider* field, and #226 hardened the transport layer against SSRF and path injection. What no layer currently checks is the trustworthiness of the document *content* and its *origin*: any well-formed profile document from any URL is written into the local store and becomes executable agent instruction. The schema validation built in #575/#585 runs on the HTTP validate/write endpoints, but `cao install` does not route through it.

This issue proposes the next layer on top of #226's transport hardening: provenance, content screening, and trust policy at install time. Scope is the profile install path first; extending the gate to the skills install path is a planned follow-up (see Proposed solution).

## User Stories

- As a **CAO user installing a profile from a URL**, I want the install to record where it came from and its content hash so that I can audit what is in my agent store and detect if a source starts serving different content.
- As a **security-conscious operator**, I want install-time screening for known prompt-injection and instruction-override patterns so that a hostile profile is flagged before it becomes agent instruction, not after it has run.
- As a **team admin deploying CAO in a locked-down environment**, I want to configure an install policy (e.g. block or require confirmation for unknown origins) so that only vetted origins can add executable profiles to my fleet.
- As a **CI/automation user**, I want a non-interactive override (`--yes` / policy setting) so that trust prompts don't break scripted installs.
- As a **future marketplace user (#573)**, I want the trust layer to exist beneath the plugin install surface so that marketplace growth does not outpace safety.

## Acceptance Criteria

- Every remote profile install records source URL, SHA-256 content hash, and install timestamp; visible via CLI and HTTP API
- Re-install of the same name from a different source or with a changed hash requires explicit confirmation in interactive use; non-interactive flows use `--yes` or a policy setting, and the HTTP API returns a 409-style conflict requiring an explicit override parameter
- Install-time screening runs on the full document content (body, not just frontmatter), is **cost-bounded using the same budget machinery as #585** (bounded traversal, bounded findings), and reports through the standard finding model
- Screening is explicitly an **advisory heuristic layer**: patterns are data-driven, documented for community contribution, and findings default to warning severity — it raises the attacker's cost, it does not claim to catch what pattern matching cannot
- Trust levels are assigned per source (`builtin` > `trusted-origin` (user-configured allowlist) > `unknown`) with configurable install policy
- The profile install path routes through the gate regardless of entry point (CLI or HTTP); a follow-up issue extends the gate to the skills install path
- Documentation includes a threat model summary and configuration guide

## Proposed solution

Three independently shippable phases, scoped to the profile install path:

**PR 1 — Provenance recording (Layer 1)**
- Extend the local store metadata (or a new `provenance.py` module) with source URL, SHA-256, and timestamp per installed profile
- Modify `install_service.py` to record provenance on every install and detect changed-hash/changed-source reinstalls (interactive confirmation, `--yes` override, HTTP conflict response)
- Expose provenance via `cao profile list` / `cao profile show` and the profiles HTTP surface in `api/main.py`
- Pure metadata; no behavior change for existing users

**PR 2 — Install-time content screening (Layer 2, advisory)**
- New module `content_screening.py`: pattern-based scan of full document content for prompt-injection markers, embedded instruction overrides, suspicious tool-invocation directives, and encoded payloads
- Patterns shipped as data files so detections can evolve without code changes; contribution guide included
- Scan cost bounded with the same traversal/findings budget approach as the #585 validator hardening, so screening cannot itself become an amplification vector
- Framing: this is a cost-raising heuristic layer, not a complete defense — pattern-based detection cannot catch all embedded prompt injection, and the issue text should not claim otherwise. Findings default to warning; policy can escalate to block
- Findings flow through the existing `profile_validator.py` finding model

**PR 3 — Trust levels and policy (Layer 3)**
- Trust level assignment per source: `builtin` > `trusted-origin` (user-configured allowlist) > `unknown`
- Install policy configuration (e.g. `require_confirmation_for: unknown`, `block: unknown`)
- Unknown-origin installs show a visible summary of what the profile can do (roles, tools, mcpServers requested)

**Follow-up (separate issue):** extend the gate to the skills install path (`cli/commands/skills.py` / `utils/skills.py`), which today is a separate code path from the profile install service. Unifying the two install flows is deliberately out of scope here to keep each PR reviewable.

**Explicit non-goals:** cryptographic signing / key distribution (future RFC; hash + provenance covers the near-term need), runtime sandboxing (tool restrictions already cover runtime), building a marketplace (that is #573 — this is the trust layer underneath it), skills-path unification (follow-up issue).

## Additional context

**Is your feature request related to a problem? Please describe.**
The agent skill ecosystem is now a documented supply chain attack surface. Recent industry research: ToxicSkills (2026) scanned ~4,000 marketplace skills and found roughly 1 in 8 contains at least one critical flaw (malware or embedded prompt injection), with over a third affected at any severity ([summary](https://www.unite.ai/ai-agent-skills-supply-chain-security-vulnerabilities/)). Palo Alto Unit 42 documented malicious campaigns against a public skill marketplace within weeks of launch ([writeup](https://unit42.paloaltonetworks.com/openclaw-ai-supply-chain-risk/)). Orca Security demonstrated working attack vectors in an AI agent skills marketplace ([writeup](https://orca.security/resources/blog/ai-agent-skill-supply-chain-security/)). arXiv 2605.28588 shows these attacks rely on prompt injection embedded in skill documents — a class classical scanners miss (which is also why Layer 2 here is framed as advisory, not a guarantee). CAO profiles use the same markdown + YAML frontmatter document model these attacks target. #226 hardened the transport and path layer of `cao install`; content trustworthiness and origin tracking are the remaining open layers.

**Describe alternatives you've considered**
- *Cryptographic signing from day one:* strongest guarantee, but requires key distribution and publisher identity infrastructure that doesn't exist yet; hash + provenance + screening delivers most of the practical value at a fraction of the scope. Deferred to a future RFC.
- *Runtime-only defenses (tool restrictions):* already exist in CAO, but they limit blast radius after a hostile profile is installed rather than preventing installation. Install-time is the cheaper choke point.
- *Rely on external registry scanning:* only helps for content installed from registries that scan; CAO installs from arbitrary URLs, so the client-side gate is the only layer that covers every source.
- *Route `cao install` through the existing #575/#585 schema validator:* worth doing, but schema validation answers "is this well-formed?", not "should I trust this?" — provenance and content screening are orthogonal to schema shape.
- *Do nothing until a marketplace exists:* #573 proposes exactly that marketplace surface; landing the trust layer after the distribution surface repeats the mistake documented in the Unit 42 writeup.

**Additional context**
Related: #226 (install transport/path hardening — this builds the next layer on it), #573 (plugin install/marketplace surface — this issue is the trust layer beneath it), #575 / #585 (profile validator service and write endpoints — the finding model Layer 2 reuses).

I'm happy to take this on — I built the profile validator service (#575) and write endpoints (#585), so the extension points are familiar territory. Filing for discussion on scope and phasing before starting PR 1.

Contributor guide

Open the contributing guide

Research direction

Start with install_service.py and the local store metadata, then inspect api/main.py and the existing profile_validator.py finding model from #575/#585. Work through the three proposed phases in order, using the stated acceptance criteria to define provenance visibility, bounded advisory findings, trust policy behavior, and documentation coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, cli, documentation, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.