PostHog / PostHog/posthog

feat(signals): forward resolve notes to scouts, research and implementation runs

Open
#93,067 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement feature self-driving
Dominant language
Python
Stars
39.9k
Forks
3.4k
Avg merge
6h 51m
Merged PRs (30d)
232

Description

Is your feature request related to a problem?

When someone resolves an inbox report with a note, that note is stored but never forwarded. A dismissal note reaches the authoring scout as a report_dismissal steering note at cold start, and reaches the research run through load_research_steering. A resolve note does neither.

The resolve flow posts to the same POST /signals/reports/<id>/state/ endpoint as dismiss, with state="resolved", a resolve reason (fixed_outside_posthog, pr_merged, already_fixed, other), and an optional dismissal_note. The backend writes a dismissal artefact and then calls forward_dismissal_note, but _FORWARDED_STATUS_VERBS in products/signals/backend/dismissal_notes.py has no entry for RESOLVED, so _describe drops the report before a note is written. The scout prompt (_SCOUT_NOTES in scout_harness/prompt.py) states this explicitly: "A resolved report never reaches you this way."

The reasoning was that resolving means the report did its job, so there is nothing to steer on. In practice, people type useful context into a resolve note: what the fix was, where it landed, why it was fixed outside PostHog, what to watch for, or a caveat the next run should know. Today the only path for that text is dismissal_note on the reports API, which an agent sees only if it happens to retrieve that report. The research run on a recurrence gets the resolved report's title and summary as "Previously resolved report" context, but not the note. The implementation run never sees it.

Describe the solution you'd like

Forward resolve notes the same way dismissal notes are forwarded, so scouts, research runs, and implementation runs can weigh the text, decide whether to act on it, and record anything durable in the scratchpad.

Backend: forward on resolve
  • Add SignalReport.Status.RESOLVED to the forwarded set in dismissal_notes.py with the verb resolved. Only forward when there is note text (a resolve reason code alone carries nothing to steer on; wrong_repo stays the one no-prose exception).
  • Add a new origin SignalScoutNote.Origin.REPORT_RESOLUTION (report_resolution) rather than reusing report_dismissal. A resolve is not a verdict that filing the report was wrong, the prompts frame report_dismissal as exactly that, and the inbox labels it "From a dismissal". A separate origin lets prompts, the notes UI, and telemetry treat it correctly. Adding a TextChoices value on a CharField with choices needs a small AlterField migration.
  • Keep the refund path unchanged. refund in views.py calls the same forwarding helper and today relies on the forwarding path dropping a report that a refund left RESOLVED (merged-PR case), pinned by test_no_note_when_a_refund_leaves_a_merged_pr_report_resolved. Make that explicit instead of implicit: for example an include_resolved flag that the two state endpoints pass and refund does not, or have the helper key on the caller's intent rather than only on the resulting status.
  • Bulk resolve already funnels through _forward_dismissal_note(reports=transitioned), so one note per scout for a bulk resolve comes for free once RESOLVED is forwarded.
  • Note content (_build_note_content): keep the shape (subject, reason code, quoted note, closing guidance) but write resolve-specific guidance. It should say the report was resolved, that the finding was real and a fix shipped, and ask the reader to record that a fix shipped and when so a recurrence is distinguishable from the original, fold anything durable into an already_addressed: or pattern: entry, and keep watching the topic rather than stop filing.
  • Same DERIVED_NOTE_TTL, same _may_steer_scouts authorization. The dismissal reasoning applies unchanged: the text already reaches run context through dismissal_note on the reports API, so the extra API key scopes are not required on top of the RBAC and team-scope legs.
Scouts
  • Update _SCOUT_NOTES in scout_harness/prompt.py: remove the "A resolved report never reaches you this way" paragraph and add a report_resolution bullet next to report_dismissal, report_discussion. It should say a resolve means the finding was real and done, that the note often carries where and how the fix landed, and that the scout should record a fix shipped (with the date) and keep watching for a recurrence rather than treat it as a reason to stop filing.
  • Update the _DISMISSAL_CONTEXT guidance if it should mention that dismissal_reason and dismissal_note on a resolved report describe the fix rather than a rejection.
Research run
  • load_research_steering already reads every origin, so a report_resolution note reaches the research prompt as soon as it exists. Add report_resolution to _DERIVED_ORIGINS in report_steering.py so the origin is classified consistently, and add a resolution_notes_attached counter to ReportSteering alongside dismissal_notes_attached, reported on the steering telemetry event in temporal/agentic/report.py.
  • Update _RESEARCH_NOTES_HEAD copy, which currently says the derived notes carry what a person said when they "dismissed, discussed, or rated" an earlier report.
  • On a recurrence, the grouping activity creates a fresh report linked to the resolved one via a related_to artefact, and research receives the resolved report's title and summary through _render_resolved_report_context in report_generation/research.py. Also pass the resolved report's latest resolve note (reason and text from its newest dismissal artefact) into that block, since this is the single most relevant place for it: the run is deciding whether this is a regression of that fix.
Implementation run
  • load_report_steering excludes all derived origins because they quote report content (title and reviewer text built from raw product data) into a run that can push a PR. Include report_resolution in the implementation run's read set while keeping the other derived origins excluded.
  • To keep the injection surface the same as today, render the resolution note body without the report title: report id, reason code, and the human-typed note only. The note text is authored by someone who passed the steer-scouts gate, which is the same trust level as a human note. Update the report_steering.py module docstring and _IMPLEMENTATION_NOTES_HEAD to describe this.
  • This is the one design point worth a reviewer's eye before implementation: whether including a human-authored resolve note, minus the title, in a PR-pushing run is acceptable. If not, the fallback is to forward to scouts and research only and leave implementation on human notes.
Other consumers of the origin enum
  • scout_harness/views.py notes list: add report_resolution to the origins withheld from callers without report read access.
  • scout_harness/tools/notes.py module docstring and leave_note comments that enumerate the derived origins.
  • products/signals/frontend/inbox/logics/scoutNotesLogic.ts: add a label, for example "From a resolution".
  • Regenerate OpenAPI types with hogli build:openapi so the origin enum in products/signals/frontend/generated/api.schemas.ts and services/mcp/src/api/generated.ts picks up the new value.
  • products/signals/backend/models.py: the SignalScoutNote docstring lists the derived writers and says "not resolve". Update it.
  • products/signals/ARCHITECTURE.md: mention the resolve forwarding path where the resolve transitions are described.
Tests
  • test_dismissal_scout_notes.py::test_only_transitions_that_judge_the_report_are_forwarded: the resolve case changes from 0 notes to 1. Add cases for a resolve with no note (0 notes) and for a bulk resolve writing one note per scout.
  • A test that the resolve note body carries the resolved verb, the reason code, the quoted note, and no report title.
  • test_no_note_when_a_refund_leaves_a_merged_pr_report_resolved must keep passing.
  • test_report_steering.py: research steering carries a report_resolution note and counts it; implementation steering carries report_resolution but still excludes report_dismissal, report_discussion, and report_feedback.
  • Research prompt test: the "Previously resolved report" block includes the resolve reason and note when present.

Describe alternatives you've considered

  • Reuse the report_dismissal origin with a resolved verb. Least plumbing, but every prompt and the inbox UI would present a resolve as a dismissal, and the scout prompt tells scouts a dismissal is the one thing that should make them stop filing. That is the opposite of what a resolve note means.
  • Rely on dismissal_note on the reports API. This is the status quo. It only reaches an agent that happens to retrieve that report, and never reaches the implementation run.
  • Forward to scouts and research only. The safe fallback if the implementation-run question above lands on no.

Additional context

Relevant code:

  • products/signals/backend/dismissal_notes.py (_FORWARDED_STATUS_VERBS, _describe, _build_note_content)
  • products/signals/backend/views.py (state, bulk_state, refund, _forward_dismissal_note, _transition_report_state)
  • products/signals/backend/report_steering.py (_DERIVED_ORIGINS, load_report_steering, load_research_steering)
  • products/signals/backend/scout_harness/prompt.py (_SCOUT_NOTES, _DISMISSAL_CONTEXT)
  • products/signals/backend/report_generation/research.py (_render_resolved_report_context)
  • products/signals/backend/temporal/grouping.py (recurrence of a resolved report)
  • products/signals/backend/models.py (SignalScoutNote.Origin)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with products/signals/backend/dismissal_notes.py, views.py, report_steering.py, and models.py, then read the named prompt, research, and frontend notes files. Run the listed dismissal and report steering tests first. Done means resolve notes are forwarded with a distinct origin to the intended scout, research, and implementation consumers, with migrations, generated types, prompt/UI updates, and all specified tests passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, python, typescript
Domain
api, backend, frontend, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.