pi: prompt drift behind enforcement — agents self-censor rm, stall on schema lag, and trip fence false-positives

Summary

End-of-run agent remarks from flag-driven pi runs (wizard-use-pi-harness=true, gpt-5.4/low, build=prod, posthog.com/dev/ci runs excluded, July 9–14) cluster around three causes. All three trace to the same asymmetry: pi got the Anthropic path's restrictions without its affordances, and the pi prompt has drifted behind pi enforcement.

Remark rates over the last 48h of prod runs:

Data: insight mTiRZTvj (Remarks: flag-driven pi runs).

Root causes

1. Scoped rm is allowed since #834, but the prompt still says it isn't

PR #834 (7e030af5, released v2.41.0 on 2026-07-10) added isScopedFileRemoval() (src/lib/agent/runner/harness/pi/security.ts:109, carve-out at :312) so a plain rm [-f] <file inside workdir> passes the fence. But no prompt line was added: PI_RUNTIME_NOTES (src/lib/agent/runner/harness/pi/index.ts:62) still says bash is "ONLY for install/build/typecheck/lint/format commands the project itself defines" and "If a bash command is blocked, do NOT retry it." The context-mill conclude step asks the agent to remove .posthog-events.json; pi has no delete tool; so the model concludes deletion is impossible without ever trying rm and remarks about it nearly every run.

The stale comment at pi/index.ts:641 ("pi's rm is fence-blocked, so the agent can't") predates #834 and is now wrong.

No user-visible damage today — host-side cleanup (pi/index.ts:646) removes the plan file after the run. The cost is wasted agent turns and remark noise.

2. Schema-lag guidance is missing from the shared prompt

Agents don't know that newly instrumented events won't appear in read-data-schema until ingested, so they stall/hedge on dashboard + insight creation. This is harness-agnostic (it shows up on anthropic too, just 10x less often in remarks) and belongs in shared guidance, not pi-only notes.

3. Fence false positives (pi-only pain)

wizardCanUseTool (src/lib/agent/agent-interface.ts:459) blocks:

• any command containing ( or ) via DANGEROUS_OPERATORS = /[;$()]/ — kills every Next.js App Router path (app/(main)/page.tsx`) in lint/format args
• pnpm --filter client typecheck — --filter sits where the script name is parsed
• npx expo install — expo isn't run/exec, and "expo install" doesn't start with an allowlisted script
• npm test / npm start — not in SAFE_SCRIPTS
• xcodebuild — not in PACKAGE_MANAGERS

The Anthropic path never feels this: Bash is in allowedTools with permissionMode: 'acceptEdits', so the SDK auto-approves it (OS sandbox + YARA hooks protect instead) and canUseTool is only consulted on permission escalation. Pi checks every call through the fence (pi/security.ts:299).

Proposed fixes

• Fix 1 (small, high value): add a PI_RUNTIME_NOTES line stating plain rm <relative-file-inside-project> is allowed (call out .posthog-events.json cleanup as the intended use); update the stale comment at src/lib/agent/runner/harness/pi/index.ts:641.
• Fix 2 (shared): add a schema-lag rule to getWizardCommandments() (src/lib/agent/commandments.ts): newly instrumented events won't be in read-data-schema yet — proceed with dashboard/insight creation anyway.
• Fix 3 (fence quality, follow-ups in src/lib/agent/agent-interface.ts):
  • make the DANGEROUS_OPERATORS check ignore (/) inside quoted path arguments
  • recognize pnpm --filter <pkg> <script> (validate the script name after the filter)
  • recognize npx expo install
  • decide deliberately on test/start (likely keep blocked — but say so in the prompt so agents don't burn turns discovering it)

Created with PostHog Code

Start with PI_RUNTIME_NOTES and the stale comment in src/lib/agent/runner/harness/pi/index.ts, then read getWizardCommandments() in src/lib/agent/commandments.ts and wizardCanUseTool in src/lib/agent/agent-interface.ts. Trace the existing fence rules before changing behavior. Done means scoped rm and schema-lag guidance are accurately described, the listed false positives are addressed or deliberately documented, and stale comments no longer contradict enforcement.