theam / theam/facility

Per-turn autonomy limits: a project policy ceiling and a propose-only mode

Open
#400 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
71
Forks
64
Avg merge
15h 38m
Merged PRs (30d)
66

Description

In short: every turn gets a maintainer token, whatever the turn is for. I propose a per-project ceiling on what a turn may do, a propose-only mode where the agent commits but cannot push or open a PR, and a per-turn risk check that can only tighten within that ceiling. The default stays exactly as today. Related: #392 asks for per-agent permissions; this is per-turn and compatible with it.

The problem

Line numbers are from main at d50f3c4.

Every agent prompt says (services/api/src/turns/dispatcher.ts:772): "You have full workspace, network, Docker, browser, git, and GitHub maintainer access." The token behind that is minted at services/api/src/github/client.ts:92-94 with { installation_id, repositories } and no permissions. Nothing between the budget check (dispatcher.ts:145) and the mint (:152) asks what the turn is for. A README typo fix and a database migration get the same token.

The docs call this the contract (ROADMAP.md:19-20, README.md:91, apps/docs/docs/reference/security.md:25-26). The only policy code today decides who may start an agent from a GitHub comment (services/api/src/agents/github-command-policy.ts:32,49). Nothing governs what a started turn may do.

Before merge, a turn that goes wrong (a mistaken agent, or a prompt injected through an issue or comment) can push to any unprotected branch, open pull requests, run CI and spend budget. Branch protection (ROADMAP.md:38) covers none of that.

I want to say, per project: "a turn that looks risky, or runs under this agent, works in the worktree but does not push or open a PR until a human looks." Enforced in code, not in the prompt.

Proposal

1. A policy ceiling per project. Deterministic, no model, reviewed as code. A policy section in .facility.yml (ROADMAP.md:35 makes it the only manifest; ProjectManifestSchema at services/api/src/workspaces/project-environment.ts:47-71 is .strict() and would gain the key):

policy:
  default_mode: full
  max_mode:
    scheduled-audit: propose-only

Only these two rules, because before a turn runs nothing knows which files it will touch, and branches already belong to GitHub branch protection. Path rules could be checked after the turn from the changed-file list TurnGitEvidenceService stores; that is a later phase.

2. Modes. full is today's behaviour. propose-only: the agent commits in the worktree but cannot push or open a PR. When it finishes, Facility opens a "proposal ready" attention item with the commit and changed-file lists git evidence already records. Approving it queues a follow-up turn in full mode ("Proposal approved. Push the branch and open the pull request."), reusing the retryAttention flow (services/api/src/stories/service.ts:902). If the workspace is deleted first, the item is resolved as lost. This needs a mode column on turns, a mode on queueMessage, and a propose-only variant of the prompt line at dispatcher.ts:772.

3. A per-turn risk check, behind an interface. Runs before the token is minted. Input: story title, message, agent, policy. Output: { level: low|medium|high, confidence, reasons }. It can only pick a mode at or below the ceiling. Low confidence fails closed, or asks a human through an attention item if the project prefers. The decision is written as a turn.policy event next to turn.phase, so it shows in Run details. Default implementation: keyword heuristics, no model. An external classifier that returns a level and a confidence could sit behind the same interface later, off by default.

Enforcing propose-only. Two options.

  • (A) Narrow that turn's token. Pass permissions: { contents: "read", issues: "read", pull_requests: "read" } to POST /app/installations/{id}/access_tokens. GitHub then refuses push and PR creation; clone, fetch and read-only gh keep working. One optional field on the token factory (client.ts:47-50, :92-94), plumbed through services/api/src/github/workspace-credentials.ts:93. Environment setup (project-environment.ts:235, :572, :588) also runs read-only, so a project whose setup script pushes cannot use propose-only. I have not tested a narrowed token. One call against any installation, then git fetch and git push with it, would settle it.
  • (B) Keep the token for setup, strip it for the agent. Remove GH_TOKEN, GITHUB_TOKEN and FACILITY_GITHUB_CREDENTIALS from the agent's environment at dispatcher.ts:297. No API change and no contract text change. But the agent loses GitHub read access for the turn, and the start script still runs with the full token (project-environment.ts:568-574), so any service it leaves running keeps it.

I lean to (A): GitHub enforces it and the agent keeps read access. (B) is the fallback if you do not want any token narrowing. The git credential helper cannot do this either way: git does not tell a helper whether the operation is a fetch or a push (runner/facility-git-credential.mjs:22 checks only host and path).

What changes. Unchanged: the merge boundary (ROADMAP.md:38), the app's own permissions, no per-agent profiles, and full by default, so a project without a policy section behaves exactly as today. New: a per-project ceiling and a per-turn mode. Under (A), some turns get a read-only token, so ROADMAP.md:36-37, README.md:91, client.ts:68-71 and security.md:25-26 each need one sentence saying so. Under (B) no text changes.

Alternatives I considered
  1. #392 alone. Per-agent permissions are static: every turn of an agent gets the same rights, and it needs profile management, which ROADMAP.md:36-37 rules out. This proposal is compatible and narrower. Under (A) it uses the same permissions parameter, per turn. The per-agent max mode covers most of what #392's manifest declaration would.
  2. Do nothing. The merge boundary protects the default branch. Everything before merge stays open to a maintainer token.
  3. Filter MCP tools. Pushes go through git and gh inside the workspace (README.md:89), not through MCP.
Phases
  1. Ceiling, propose-only, mode on turns, the "proposal ready" item and its approval flow, with (A) or (B). No risk check yet.
  2. The risk-check interface, the heuristic default, the turn.policy event and its Run details case (services/api/src/turns/activity.ts:66,82).
  3. An external assessor behind config, off by default.

Each with the tests AGENTS.md:5-7 asks for (success, denial, malformed, cross-tenant), plus a regression test that fails if a propose-only turn can push.

Questions for maintainers

CONTRIBUTING.md:10 asks for the contract to be agreed first, so before a PR:

  1. Is a per-turn read-only token (A) acceptable under ROADMAP.md:36-37, or should Phase 1 use (B)?
  2. Should the ceiling live in .facility.yml or in projects.settings?
  3. On approval: a follow-up turn, or Facility pushes the branch itself and only withholds the PR?
  4. (Phase 2) On low confidence: fail closed, or ask a human?
  5. Would you take Phase 1 as a PR once these are answered? I can do it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with CONTRIBUTING.md:10, ROADMAP.md:35-38, and the listed dispatcher, token client, manifest, credential, and story-service entry points to resolve the Phase 1 design choices. Done means maintainers agree on token strategy, manifest location, approval flow, and the required success, denial, malformed, cross-tenant, and push-regression tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, github, typescript
Domain
authorization, backend-api-design, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.