[RFC] Towards Self-Evolving Agents: Interactive Instruction Distillation (/learn) and Rule Metabolism for AGENTS.md

[RFC] Towards Self-Evolving Agents: Interactive Instruction Distillation (/learn) and Rule Metabolism for AGENTS.md

1. Executive Summary & Core Philosophy

As coding agents tackle increasingly complex, multi-week software projects, they face a fundamental evolutionary bottleneck: The Amnesia-vs-Accumulation Dilemma.

1. Amnesia (The Groundhog Day Problem): Without durable learning, each new session starts from scratch. Developers find themselves repeatedly steering the agent on the same repository quirks, local sandbox boundaries, tool arguments, and testing gates.
2. Accumulation (The Memory Swamp & Unchecked Degradation): Naive instruction systems that only accumulate text without an explicit promotion/retirement lifecycle risk Instruction Drift and Semantic Bloat—obsolete workarounds, conflicting rules, and unverified AI speculation can gradually degrade reasoning quality. Recent empirical research (e.g., Microsoft Research's SkillOpt, arXiv:2605.23904) demonstrates that ungated, unsupervised self-edit loops can rapidly degrade model performance (collapsing from 0.554 to 0.026 in single-seed benchmarks) by overfitting to superficial patterns.

The Guiding Principle: Instruction Metabolism & Rules as Hypotheses

True agent self-evolution is not about remembering everything; it is about metabolism—treating every rule as a falsifiable hypothesis, enforcing selective extraction, progressive refinement, and the active retirement of superseded rules under human consensus.

Codex already has a sophisticated two-phase memory architecture (codex-rs/memories) with consolidation, pruning, deduplication, and stale-evidence cleanup. This RFC proposes extending that general philosophy into an explicit, user-visible, auditable lifecycle for authoritative instructions, bridging passive memories with developer workflows and AGENTS.md.

---

2. Analysis of Codex's Current Architecture (codex-rs)

OpenAI Codex already possesses a solid foundational memory harness:

• Phase 1 (Rollout Extraction): Asynchronously extracts raw_memory, rollout_summary, and rollout_slug into the local SQLite state DB.
• Phase 2 (Global Consolidation): An internal background agent uses a Git-backed workspace under ~/.codex/memories/ to maintain MEMORY.md, memory_summary.md, and local skills/.

The existing consolidation path already performs meaningful lifecycle work: incremental updates, no-op decisions, stale-input pruning, deduplication, and removal or rewriting of memory no longer supported by evidence. The gap is therefore not “Codex lacks forgetting.” The gap is that this lifecycle is largely passive and memory-scoped, while authoritative operating instructions remain outside an equivalent user-directed promotion/revision flow.

The Critical Structural Gaps

1. Isolation from Authoritative Instructions (AGENTS.md)

   • codex-memories-write updates the memory workspace, but does not intentionally promote validated session learnings into global ~/.codex/AGENTS.md or repository-level ./AGENTS.md.
   • Project constraints, architectural invariants, and team boundaries therefore remain outside the current memory consolidation lifecycle unless a developer edits them separately.

2. Absence of In-Session Human-Agent Consensus (/learn)

   • Memory consolidation is primarily asynchronous and startup-triggered.
   • Developers cannot intentionally run a distillation pass immediately after a difficult debugging session, repeated correction, or architectural decision and review the proposed durable rule changes before they are applied.

3. Lack of an Explicit, User-Visible Lifecycle for Authoritative Instructions

   • AGENTS.md rules do not currently have a first-class interactive lifecycle such as ADD, NARROW, REPLACE, or RETIRE driven by session evidence and confirmed by the user.
   • Without such a lifecycle, teams must manually reconcile stale, overlapping, or overly broad instructions over time.

---

3. Proposed Architecture & System Design

                     ┌────────────────────────────────┐
                     │    Active Codex CLI Session    │
                     │  (User corrections & debug)    │
                     └───────────────┬────────────────┘
                                     │
                 User calls `/learn` │ (or Agent suggests at turn completion)
                                     ▼
        ┌────────────────────────────────────────────────────────┐
        │        Interactive Distillation Sub-Agent              │
        │   (memory/instruction distillation pass)               │
        └────────────────────────────┬───────────────────────────┘
                                     │
                  ┌──────────────────┴──────────────────┐
                  ▼                                     ▼
     [ Rule Metabolism Engine ]            [ Epistemic Layering ]
     - Add: Net-new invariant              - Workspace: `./AGENTS.md`
     - Narrow: Restrict broad rule          - Global: `~/.codex/AGENTS.md`
     - Replace: Supersede old pattern      - Skill: `skills/<name>/SKILL.md`
     - Retire: Remove obsolete rule        - Memory: `MEMORY.md`
                  │
                  ▼
   ┌─────────────────────────────────────────────────────────────┐
   │            Human-in-the-Loop Review & Staged Diff           │
   │  "Proposed update to ./AGENTS.md (-2 lines, +3 lines): [Y/n]"│
   └──────────────────────────────┬──────────────────────────────┘
                                  │ (Approved)
                                  ▼
   ┌─────────────────────────────────────────────────────────────┐
   │         Atomic Mutation to Local/Global Instruction File    │
   └─────────────────────────────────────────────────────────────┘

3.1 Interactive Distillation Protocol (/learn / /distill)

• Trigger: Invocable by the developer via /learn (or suggested by Codex upon resolving a multi-turn error).
• Scope Extraction: Inspects the active turn transcript and execution diff for:
  • Explicit user corrections and workflow preferences.
  • Validated failure shields (Symptom → Root Cause → Fix → Verification).
  • Platform/environment constraints (e.g. TCC permissions, required flags, build timeouts).
  • Architectural invariants or contribution rules that have become stable enough to deserve promotion.

The command defaults to a review-first flow: distill candidates, show their proposed destination, lifecycle operation, and review criteria, and only write after explicit user approval.

3.2 The Four Operations of Rule Metabolism & Rules as Hypotheses

Every authoritative-rule candidate is treated as a falsifiable hypothesis rather than an unquestioned permanent decree, and is classified into one of four discrete lifecycle operations:

1. ADD: Introduce a verified, net-new constraint or workflow invariant. (Optionally tagged with a falsifiable_condition and review_after timestamp).
2. NARROW: Restrict the scope of an existing rule found to be overly broad.
3. REPLACE: Atomically supersede an older, sub-optimal rule with a validated newer approach.
4. RETIRE: Remove an obsolete rule whose underlying conditions no longer apply (e.g., when an upstream CLI bug is fixed or a library migration completes).

These operations are intended as an auditable interaction model, not necessarily a permanent on-disk schema. The important property is that Codex must distinguish “add another instruction” from “this evidence changes, restricts, or invalidates an existing instruction.”

3.3 Epistemic Layering & Fenced Machine Sections

Not every useful lesson belongs in AGENTS.md, and machine-distilled text should not corrupt human-authored architectural prose.

• Ephemeral Episode → Kept in local session context only; discarded on session close.
• Durable but Non-Authoritative Knowledge → MEMORY.md / memory artifacts.
• Workspace Invariant / Team Operating Rule → ./AGENTS.md (version-controlled with project code).
  • Fenced Block Isolation: Distilled instructions can land in a dedicated, clearly delimited block (e.g., <!-- CODEX-LEARNED:START --> ... <!-- CODEX-LEARNED:END -->) subordinate to hand-authored rules to preserve clean structural boundaries.
• Global Developer Preference / Operating Rule → ~/.codex/AGENTS.md.
• Reusable Multi-Step Procedure → Synthesized into a dedicated skills/<name>/SKILL.md.

3.4 Variance-Aware Evaluation vs. Deterministic Constraints

• Deterministic Operational Constraints (e.g., specific CLI flags --network-timeout 60, required path prefixes, formatters): Verified directly by the immediate tool execution/test pass during the session.
• Probabilistic / Behavioral Strategy Rules (e.g., prompt reframing, high-level reasoning heuristics): Treated as empirical hypotheses; when evaluated, should be checked across multiple runs ($k=5$) against a golden regression suite rather than single-run stochastic noise.

3.5 Standing Surface Budget & Adherence-Probed Capacity Gating

Every review of an instruction file inherently suffers from build-bias: rules feel like assets, leading humans and agents to continuously accumulate instructions while rarely retiring older ones. However, as the standing instruction surface expands, model adherence to any individual rule steadily degrades.

To turn retirement into an unavoidable, first-class outcome, the architecture incorporates three complementary mechanics:

1. The Forced-Choice Moment (Trade-Off Friction):

   • When a newly proposed ADD would breach the active standing surface budget, the distillation flow halts silent addition and enforces an explicit trade-off prompt:

     "Standing surface budget exceeded. Promotion requires an explicit RETIRE or MERGE of existing rules to make room for ADD."

   • This friction breaks the hoarding instinct and ensures retirement is a required operational step rather than an optional chore.

2. Adherence-Probed Capacity Gating (Empirical Evidence over Task Breadth):

   • Rather than allowing an agent to arbitrarily self-expand its budget based on workspace task breadth (which conflates what a user wants loaded with what the model actually follows, effectively letting the auditor grade its own homework), the capacity ceiling is bounded by measured adherence evidence.
   • The workspace probes multi-rule compliance with all standing rules loaded jointly. If empirical probes demonstrate quiet rule-dropping under load, the budget ceiling is hard-locked, mandating the pruning of mediocre rules before new invariants can land.

3. Path-Scoped & Skill Offloading:

   • Domain-specific constraints (e.g., prose styling, database migrations, specific framework protocols) are offloaded to directory-scoped rules or on-demand skills, keeping the root AGENTS.md strictly minimal regardless of user versatility.

3.6 Fail-Closed Anchor Guard & Interactive Semantic Re-Anchoring

A subtle failure mode in real-world human-agent co-editing is Anchor Drift leading to Zombie Rules:

• Developers frequently hand-edit AGENTS.md between sessions (rephrasing rules, moving lines, or tweaking formatting).
• When /learn subsequently attempts a REPLACE or RETIRE operation, the verbatim target text no longer matches.
• A naive implementation might silently fall back to ADD into the fenced machine section to avoid a hard error. This reports "Applied", but leaves the mutated old rule intact alongside the new rule, creating contradictory duplicates and stealthily inflating the standing surface.

To prevent this silent degradation, the engine enforces three safeguards:

1. Strict Fail-Closed Invariant: REPLACE and RETIRE must never silently degrade into ADD. If a target rule cannot be anchored with certainty, the mutation must fail-fast rather than creating a duplicate.
2. Interactive Semantic Re-Anchoring: When an exact verbatim match fails, the engine runs a fuzzy/semantic similarity search across AGENTS.md and surfaces drifted candidate lines for explicit user confirmation (Supersede this drifted line? [Y/n/select/abort]).
3. Machine-Block Stable Identifiers: Rules generated in the machine-managed fenced block carry lightweight stable identifiers (e.g., <!-- id: py-formatter -->), ensuring unambiguous targeting even if explanatory text inside the block has been lightly modified.

3.7 Grounded Retirement vs. Gated Specialization (Preventing Never-Supersession)

A critical failure mode in automated rule metabolism is the Never-Supersession / Leaky GC Trap:

• To avoid accidentally deleting valid instructions (Wrong-Supersession), an agent might naively default to synthesizing hierarchical branches (Default vs. Exception) whenever new evidence overlaps with an existing rule.
• However, merge-by-default merely trades wrong-supersession for never-supersession: genuinely obsolete rules are never retired; they simply turn into perpetual exception branches, accumulating cognitive bloat and directly undermining the standing surface budget.
• Furthermore, dormant exception branches create continuous attentional noise in the LLM context and risk accidental reactivation if the agent hallucinates or misidentifies environmental edge cases, actively impeding codebase toolchain migrations.

To ensure physical deletion remains a safe, decisive, and active primitive, the engine enforces three architectural invariants:

1. Ground-Truth Evidentiary Retirement (Workspace Scope):

   • At the workspace level (./AGENTS.md), retirement is not guessed from prose similarity. Instead, it is anchored to verifiable codebase ground truth.
   • When a session provides affirmative proof that an old dependency or configuration has been removed (e.g., .flake8 deleted, pyproject.toml migrated to ruff, and CI lanes updated), the distillation engine must emit an unconditional RETIRE or REPLACE.
   • The engine is explicitly prohibited from hallucinating a "legacy fallback branch" when ground-truth evidence contradicts the old rule.

2. Strict Gating on Specialization:

   • Specialization into hierarchical branches (Default vs. Exception) is permitted only when the workspace demonstrates observable, coexisting heterogeneous environments (e.g., distinct sub-packages or legacy modules in a monorepo).
   • Branch Budget Penalty: Synthesized branches are not free. Each conditional exception branch is weighted with an increased penalty against the Standing Surface Budget (Section 3.5), stripping the engine of any incentive to use branching as a defensive evasion of retirement.
   • Exception Half-Life: Synthesized exception branches carry an automated review timer (review_after). If subsequent sessions show zero activation across the exception path, the branch is flagged for proactive pruning.

3. Dual-Zone Addressing: Pristine Prose vs. Fenced Machine Blocks:

   • Human-authored sections of AGENTS.md remain 100% natural, idiomatic Markdown without mandatory machine IDs or annotations, preserving natural developer ergonomics.
   • Rules generated inside machine-managed blocks (<!-- CODEX-LEARNED:START -->) carry compact stable keys (e.g., <!-- id: linter-rule -->). This gives the agent deterministic targets for physical RETIRE / REPLACE mutations, eliminating reliance on fragile string-distance heuristics and avoiding rubber-stamp prompts.

3.8 Generation-Verification Decoupling & Deterministic Tripwires (Resolving the ID Dilemma)

A fundamental tension in instruction lifecycle design is the State Authority & Tripwire Trade-off:

1. The ID & Shadow-Store Trap (Zombie Resurrection): If the system enforces machine stable IDs backed by an external consolidation database or shadow memory store, updates fail loud when an ID is missing. However, the reverse path fails silently: when a human developer deletes a rule line by hand in Git, the shadow store still holds the entry. An audit enforcing that "every pre-existing entry has a successor" treats the deletion as an accidental drop and silently re-materializes (resurrects) the dead rule on the next consolidation pass.
2. The Pure-LLM Synthesis Trap (Quiet Smoothing): Conversely, if the system drops IDs and relies entirely on end-to-end LLM synthesis over raw Markdown, the loud failure goes quiet. When faced with a drifted or conflicting rule, an unconstrained model will politely emit a cleanly formatted block retaining both variants and report done. Removing the ID eliminates the tripwire, not the drift.

To resolve this without forcing line-level IDs into human prose or surrendering Git as the sole ground truth, the architecture decouples semantic proposal from physical verification:

[ Transcript / Evidence ]
           │
           ▼ Phase 1: Semantic Proposal (LLM Generative Pass)
┌─────────────────────────────────────────────────────────────┐
│ Propose Structured Patch:                                   │
│ - Target Block: Verbatim snippet of old rule to supersede   │
│ - Mutation:     REPLACE / RETIRE / ADD                      │
│ - Rationale:    Codebase evidence (files, diffs, configs)   │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼ Phase 2: Deterministic Patching & Tripwire (Code Tool)
┌─────────────────────────────────────────────────────────────┐
│ Deterministic Patch Engine (Exact-Block Anchor Check)        │
│ - Target Block exactly matches active file?                 │
│   ├── YES ──► Apply atomic in-place mutation (Commit)       │
│   └── NO  ──► FAIL LOUD: "Anchor Mismatch: Target drifted"  │
│               (Halt before write; no silent LLM smoothing)  │
└──────────────────────────────┬──────────────────────────────┘
                               │
                               ▼ Phase 3: Post-Synthesis Invariant Guard (Static / Budget)
┌─────────────────────────────────────────────────────────────┐
│ Structural & Adherence Invariant Guard                      │
│ - Reject net complexity or rule-count expansion on REPLACE  │
│ - Run binary opposition check for coexisting contradictions │
└─────────────────────────────────────────────────────────────┘

1. File-as-Authority (Zero Zombie Resurrection):
   There is no external database or shadow store acting as an independent source of truth. The Git-tracked Markdown file is the sole ground truth. If a human deletes a rule line in Git, it is permanently gone; the next session reads the file as-is and cannot resurrect a phantom rule.

2. Deterministic Content-Addressed Patching (The Non-ID Tripwire):
   The LLM is strictly prohibited from arbitrarily rewriting whole files. Instead, mutations must be executed through a deterministic tool requiring an exact TargetBlock quotation:

   • If the human hasn't touched the block, the exact byte match succeeds and applies the patch in a 1-click flow.
   • If a human hand-edited the line (causing drift), the patch engine throws an immediate, hard exception (Fail Loud: TargetContent not found). The model is blocked before touching the disk and is forced to re-read the human's modified text, restoring the tripwire without requiring line-level <id> tags.

3. Post-Synthesis Invariant Assertions:
   A lightweight static validator inspects the staged Git diff:

   • If an operation declared as REPLACE yields an expansion in rule surface or fails an opposition probe (detecting that old and new policies were merely concatenated), the patch is rejected fail-closed before landing.

---

4. Proposed User Experience (CLI Walkthrough)

$ codex
> [User]: Fix the network timeout when calling the internal build cluster on macOS.
> [Codex]: ... (triages sandbox flags, discovers missing `--network-timeout 60` and TCC bypass, fixes issue and verifies) ...
> [User]: /learn

[Codex Distillation Engine]
Analyzing session transcript (Turn 1 to Turn 7)...
Checking standing surface budget (Current: 38/40 rules, 4.8KB / 5.0KB)...

Proposed Rule Updates:

1. [Project Rule -> ./AGENTS.md] (ADD)
   Operation: ADD (Fenced Block)
   Content:   When invoking internal cluster builds, always pass `--network-timeout 60` to accommodate macOS proxy latency.
   Condition: Retire if internal cluster proxy latency is resolved upstream.
   Review:    2026-10-01

2. [Project Rule -> ./AGENTS.md] (REPLACE)
   Operation: REPLACE
   - Do not run local cluster integration tests.
   + Integration tests may be run locally if `CLUSTER_DEV_MODE=1` is set.
   [Anchor Check]: Exact anchor match verified at ./AGENTS.md:L42.

3. [Global Preference -> ~/.codex/AGENTS.md] (NARROW)
   Operation: NARROW
   Target:    "Always format Python code with flake8."
   [Anchor Drift Warning]: Verbatim target not found (user hand-edited file).
   Found drifted candidate at ~/.codex/AGENTS.md:L18 (94% match):
     - "Always format Python code with flake8 across all projects."
   Supersede drifted line L18 with new narrowed rule? [Y/n/edit]: Y
   + Default to `ruff format` across Python projects unless the repository explicitly pins a different formatter.

[Surface Budget Guard]: Proposed updates maintain headroom (39/40 rules, 4.9KB / 5.0KB).

Apply changes to instruction files? [Y/n/edit]:

---

5. Engineering Feasibility & Possible Implementation Strategy

The exact ownership boundary should follow the maintainers' preferred architecture. The proposal does not depend on introducing a specific new crate.

A plausible implementation could:

1. Reuse Existing Memory Infrastructure Where Appropriate

   • Reuse existing prompt rendering, secret-redaction, evidence hygiene, and workspace-diff helpers from codex-memories-write where those abstractions fit.

2. Add an Interactive Distillation Component

   • Given the active transcript, current effective instruction files, and relevant execution diff, produce structured candidates such as {destination, operation, evidence, proposed_diff, review_after, falsifiable_condition}.
   • This could live in codex-core, the existing memories write path, or a dedicated component depending on maintainership boundaries.

3. Add a First-Class CLI/TUI Binding

   • Register /learn and/or /distill as explicit user-facing commands.
   • Prefer a review-first flow over silent writes.

4. Generate Minimal, Transactional Instruction Diffs

   • Avoid rewriting whole AGENTS.md files when a narrow change is sufficient.
   • Preserve user-authored structure and unrelated sections.
   • Place distilled rules into delimited sections or surgically edit existing lines.
   • Reject or surface ambiguous edits instead of silently resolving them.

5. Guard Against Instruction Bloat and Authority Escalation

   • Treat “no durable change” as a valid result.
   • Require stronger evidence for global rules than workspace-local rules.
   • Detect semantic overlap with existing instructions before proposing ADD.
   • Never silently promote transient task state, speculative conclusions, secrets, or unverified fixes into authoritative instructions.

---

6. Related Work & Literature

• #34668 — Explicit, auditable promotion of operator feedback into Memories: Focuses on memory CRUD with provenance. This RFC extends this to authoritative instruction surfaces (AGENTS.md).
• #32748 — User-facing workflow retrospective / insights: Asynchronous retrospective recommendations. This RFC targets the in-session transactional /learn path.
• #21932 — Allow /init to update existing AGENTS.md: Initial repository generation vs. continuous empirical distillation.
• SkillOpt (Microsoft Research, arXiv:2605.23904): Empirically frames agent instructions as optimizable weights and proves that un-gated self-evolution leads to catastrophic degradation, validating the necessity of strict human review and held-out validation gates.
• Agent Workspace Architecture Patterns (jimy-r/agent-workspace-architecture): Documents practical patterns for heavily-instructed workspaces—specifically treating scaffolds as hypotheses and enforcing variance-floor regression checks.
• Never-Supersession & Tripwire Decoupling (@m13v, #40575): Informed Sections 3.7 and 3.8 by identifying how defensive specialization causes perpetual instruction accumulation, and how dropping IDs risks turning loud failure into quiet synthesis smoothing, directly motivating our generation-verification decoupling.

The intended composition is:

passive memory consolidation
        +
explicit memory promotion / management
        +
user-visible retrospectives
        +
interactive authoritative-rule distillation (with hypothesis gating)
        ↓
safer long-term agent adaptation

---

7. Conclusion

Codex already contains much of the hard infrastructure needed for durable learning: persisted rollouts, structured extraction, consolidation agents, a Git-backed memory workspace, pruning, deduplication, skills, and authoritative AGENTS.md instruction chains.

The missing piece is a deliberate promotion boundary between “the agent learned something from experience” and “this should now govern future work as an authoritative rule.”

An interactive /learn / /distill flow could make that boundary explicit, reviewable, and auditable:

experience → evidence → candidate hypothesis → lifecycle operation → human-reviewed diff → durable instruction

That would let Codex evolve across long-running projects without turning either memory or AGENTS.md into an unbounded accumulation of historical text.

Start by reading the existing codex-rs/memories architecture and its consolidation flow, then compare it with the proposed /learn lifecycle for AGENTS.md, MEMORY.md, and skills//SKILL.md. The RFC identifies ADD, NARROW, REPLACE, and RETIRE operations, human review, and fail-closed anchoring as core requirements. Done would require an agreed, user-visible design that preserves review and prevents stale or duplicate rules.