aws-samples / aws-samples/sample-autonomous-cloud-coding-agents

Orchestrator/ Memory: Implement review feedback memory loop so agents learn from PR reviews

Offen
#27 0 Kommentare 1 Reaktion 0 zugewiesene Personen Auf GitHub ansehen
agent-runtime orchestration
Vorherrschende Sprache
TypeScript
Sterne
143
Forks
46
Ø Merge
3 T. 10 Std.
Gemergte PRs (30 T.)
24

Beschreibung

### Component

API or orchestration

### Describe the feature

> This issue depends on #26

This addresses the "Review feedback memory loop (Tier 2)" item from Iteration 3d in the roadmap.

The agent doesn't learn from PR reviews. If a reviewer leaves comments like "use zod for validation", "return 201 for creation endpoints", "use our errorHandler middleware" - those corrections apply to every future task on that repo, but the agent has no way to internalize them. The next task on the same repo will repeat the same mistakes, and the reviewer has to leave the same comments again.

The memory system (Iteration 3b) already supports writing and reading repo-level knowledge. The webhook infrastructure (Iteration 3a) already handles inbound events. What's missing is the pipeline that connects PR review comments to persistent memory.

### Use case

I submitted a `new_task` to create an API endpoint. The agent delivered working code but without request validation, wrong HTTP status codes, and inline error handling instead of the repo's middleware pattern. I left 3 review comments. Then I submitted another task for a different endpoint on the same repo - the agent made the exact same mistakes because it had no memory of my feedback.

### Proposed solution

Use the existing webhook infrastructure to capture `pull_request_review` events and extract actionable rules into memory.

**Flow:**

1. Reviewer leaves comments on an agent-created PR
2. GitHub sends a `pull_request_review` webhook event
3. A new handler (separate from the task-creation webhook) receives the event, fetches the diff + comments
4. A Bedrock call extracts rules from the comments, classifying each as:
- **Repo-level**: "use zod for validation on all endpoints" - applies to all future tasks
- **Task-specific**: "this function needs a null check" - context for this task only
5. Rules are written to AgentCore Memory with `source_type: review_feedback` provenance
6. On the next task for that repo, `loadMemoryContext()` loads these rules during hydration - the agent starts already knowing the repo's conventions

**A real example:**

Review comments: "validate with zod", "return 201 for POST", "use errorHandler middleware"

Next task prompt includes:
```
Repository knowledge (from past reviews):
- Use zod for request body validation on all endpoints
- POST endpoints that create resources must return HTTP 201
- Use errorHandler middleware, not inline try/catch
```

The agent applies all three without the reviewer repeating themselves.

**What's we need to add:**
- A webhook handler for `pull_request_review` events (the current handler only creates tasks)
- An extraction Lambda with a Bedrock prompt to turn review comments into structured rules
- Provenance tagging on memory writes (`source_type: review_feedback`) so trust scoring (Iteration 3e) can weight these appropriately

### Other information

_No response_

### Acknowledgements

- [x] I may be able to implement this feature
- [ ] This might be a breaking change

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne mit der Überprüfung der bestehenden Webhook-Infrastruktur, des Speichersystems aus Iteration 3b und issue #26. Verfolge den aktuellen Webhook zur Aufgabenerstellung und loadMemoryContext(), und ermittle anschließend, wie pull_request_review-Ereignisse, das Abrufen von Diffs und Kommentaren, die Bedrock-Extraktion und die Herkunft von source_type zusammenhängen. Als abgeschlossen gilt die Arbeit, wenn Review-Regeln auf Repository-Ebene persistiert und in spätere Aufgaben geladen werden, während aufgabenspezifisches Feedback angemessen abgegrenzt bleibt.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
aws, github, typescript
Bereich
ai, api, backend, cloud
Issue-Typ
Feature
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Ruhig
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
38/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.