awslabs / awslabs/cli-agent-orchestrator
[Feat] Opt-in self-learning loop: outcome capture, retrospection, and instruction promotion
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 267
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 70
Description
## Overview
An **opt-in, off-by-default self-learning loop** built on the memory subsystem: agents report task **outcomes**, a built-in **retrospector** agent distills them into agent-scope memory **lessons**, and — behind a second, stricter gate — recall-reinforced lessons can be **promoted** into a delimited `## Learned Patterns` block in agent profile files. Workflows that run repeatedly over similar work items (migration pipelines, batch conversions, review loops) stop re-discovering the same lessons every run.
## User Stories
- As a **workflow operator**, I want my agent team to learn from each completed work item so that later items need fewer fix iterations.
- As a **supervisor agent**, I want to record content-free outcomes (`success`, optional score, short friction notes) so that a retrospector can distill recurring root causes into lessons.
- As a **worker agent**, I want relevant lessons from previous runs injected into my context so that I avoid known traps without being told each time.
- As an **operator**, I want lesson promotion into profile files to be dry-run by default, reviewable as a diff, capped, audited, and git-revertible so that instruction mutation stays under human control.
- As a **CAO deployer**, I want the whole feature off by default so that existing deployments are byte-identical until someone opts in.
## Acceptance Criteria
- Two new settings nest under memory: `memory.learning_enabled` (default `false`, env `CAO_MEMORY_LEARNING_ENABLED`) gates outcome capture; `memory.instruction_promotion_enabled` (default `false`, env `CAO_MEMORY_INSTRUCTION_PROMOTION_ENABLED`) gates profile mutation. Nesting rule: **promotion ⊂ learning ⊂ memory** — a disabled parent forces children off.
- Disabled behavior mirrors `memory.enabled` idioms: writes fail loud before validation (`LearningDisabledError` / `PromotionDisabledError`), reads fail silent (`[]`, 404, `{"disabled": true}` MCP payloads), read errors fail closed.
- Outcomes are content-free by contract: short labels + ≤1000-char friction notes, never transcripts/logs/secrets.
- Promotion applies **itemized per-lesson deltas** to the delimited block (never whole-block rewrites), preserves everything outside the block byte-for-byte, writes atomically, caps at 10 lessons × 400 chars, skips (never evicts) at cap, refuses built-in package profiles, and writes a content-free audit entry.
- `cao memory promote ` is dry-run by default; `--apply` required to mutate.
- Full test coverage for flag nesting, disabled gates, delta editing (incl. marker-injection rejection and corruption handling), promotion eligibility, and an end-to-end loop test.
## Proposed solution
Implementation is complete on a branch (~4,800 lines, 100 new tests, 5,378 total passing) and can be sent as one PR or split (① `CAO_HOME_DIR` override, ② outcome capture + retrospector, ③ promotion + docs) — whichever review shape maintainers prefer:
- `services/outcome_service.py` — `record_outcome`/`list_outcomes`, `workflow_outcomes` table (+ indexes) in `clients/database.py`
- `mcp_server/server.py` — `report_outcome` tool (terminal context auto-resolved from `CAO_TERMINAL_ID`)
- `api/main.py` — `POST/GET /outcomes` (write-scope gated; 404 while disabled)
- `agent_store/retrospector.md` — single-purpose profile (pattern: `memory_manager`); lessons are ordinary agent-scope memories ending with `Applies when: `, so injection/recall/lint/retention/audit all apply unchanged
- `services/learned_patterns.py` — delimited-block delta editor; `services/promotion_service.py` — plan/apply with the recall-count gate (`access_count >= 3`); `cli/commands/memory.py` — `promote` verb
- `skills/cao-learning/` — shipped skill teaching supervisor/worker/retrospector habits
- `docs/self-learning.md`, `docs/self-learning-validation.md`, plus `configuration.md`/`api.md`/`memory.md` updates and CHANGELOG entries
- Standalone enabler included: `CAO_HOME_DIR` env override for the data directory (isolated side-by-side instances; default unchanged)
## Additional context
**Is your feature request related to a problem? Please describe.**
Memory persists what agents choose to store, but nothing closes the loop from "this step failed and here is why" to "the next run starts knowing that." Repeated workflows pay the same learning cost every run.
**Evidence.** We validated the loop with a 20-package controlled A/B experiment (identical conversion tasks; learning on vs off; blind median-of-3 LLM judge): on work items where the baseline struggled (control score < 80), the learning arm won **6/6 with mean +11 points (sign test p = 0.016)**; on items the baseline already handled well the effect was neutral (−0.1). Cost: ~2× worker latency from injected lesson context. The full write-up (method, per-package table, threats to validity) ships as `docs/self-learning-validation.md`.
**Alternatives considered.**
- *Session-end auto-retrospection* (plugin on `post_kill_session`): deferred — supervisor-driven handoff keeps agent count and cost explicit.
- *Automatic promotion without a reinforcement gate*: rejected — plausible-but-wrong lessons would reach shared profiles; recall count is a signal the memory system already tracks, and operators review every promotion diff.
- *Prompt-optimizer-style full rewrites* (GEPA/ACE): rejected for profile mutation — itemized deltas on a delimited block avoid context-collapse and keep profiles human-owned outside the block.
Contributor guide
Research direction
Start by reviewing the proposed entry points in services/outcome_service.py, mcp_server/server.py, api/main.py, services/promotion_service.py, and cli/commands/memory.py, then inspect the branch's tests. Verify the nested disabled gates, outcome and promotion behavior, delta editing safeguards, audit output, and end-to-end loop against the acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend-api-design, cli, databases, devtools, documentation
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100