Scouts: remove the per-scout emit (dry-run) setting
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.9k
- Forks
- 3.4k
- Avg merge
- 7h 27m
- Merged PRs (30d)
- 222
Description
SignalScoutConfig.emit is a per-scout dry-run posture, defaulting to on. A scout should always be able to write and edit reports, so the setting has no job left to do. It is an early design relic and should come out.
Worth stating plainly: #88605 just added this toggle to the web settings form, because it was reachable through the API but not the UI. That was the right fix for the inconsistency it found, and it makes the setting's real cost easier to see. This issue removes the setting and the toggle with it. Closed #88966 as the issue behind that PR.
Why remove it rather than keep it
Three things make it worse than redundant.
It blocks the report channel, not just signals. _preflight_emit_gates in products/signals/backend/scout_harness/tools/emit.py:510 is shared by emit_finding, emit_report, and record_structured_output. A dry-run scout cannot author or edit a report at all. That contradicts what a scout is for.
It is not a preview mode, it is a shredder. Dispatch only filters on enabled (config_registry.py:58), so a dry-run scout runs in full and costs a full agent run. Then _gate_skip_result (report.py:326) returns report_id=None and persists nothing. The report the agent just wrote is discarded. Nobody can read what it would have said. So dry-run buys you the spend of a live scout and the output of a disabled one, and enabled=false already covers "do not run this".
It forces special cases across the harness. The inactivity sweep filters to emit=True and carries a comment explaining that a dry-run scout's silence means nothing (inactivity.py:185). structured_output.py:381 raises rather than record. runner.py:700 strips the schema snapshot and runner.py:973 withholds a prompt section. report.py:769 keeps a set of skip reasons that must not fan out to customers. Each is a branch that exists only because a scout might be configured to throw its work away.
What a person sees afterwards
- No mode control in scout settings or the create form, in the web app or the desktop app.
- No "Dry run" badge in the roster, the summary row, or the scout page. A scout is on or off.
- A scout that runs writes its findings and reports. There is no configuration in which it runs and discards them.
- The gates that stop output for a real reason still stop it, and still say why.
Removal surface
Backend:
- Drop the field on
SignalScoutConfig(models.py:1404) and the doc comments around it, including thestructured_output_schemacomment that says the channel requires emit. _preflight_emit_gates: drop thescout_emit_disabledbranch and itsEMIT_SKIP_REMEDIATIONentry, and remove it from_INACTIVE_SKIP_REASONS.structured_output.py: drop theif not config.emitraise in_reserve_capacity, and the dry-run clause inInvalidStructuredOutputError's docstring.runner.py: bothconfig.emitconditionals become unconditional on the schema being set.inactivity.py: drop theemit=Truefilter and its comment.prompt.py:250: the fleet roster line telling other scouts that a dry-run scout's silence is meaningless.- Serializers: the field on the read, update, create-options, and fleet-entry serializers, plus the caveat in
EmitEligibilitySerializer.can_emit's help text. - Views: the
emitmentions in the list, create, update, and destroy endpoint descriptions. admin.py:95,102:list_displayandlist_filter. These break at import if the field goes and admin does not, so they belong in the same change.
Frontend, web and desktop:
- Web:
ScoutConfigControls.tsx,ScoutCreateModal.tsx,ScoutBadges.tsx,ScoutSummaryRow.tsx,ScoutsRoster.tsxdry-run count,scoutGroups.ts,InboxWaitingForWork.tsx,scoutCreateModalLogic.ts, mocks and tests. - Desktop:
features/scouts/components/ScoutConfigControls.tsxmode select,ScoutBadges.tsx,ScoutsFleetSection.tsxdry-run count,useScoutConfigMutations.tstracked settings, the hand-rolledScoutConfig.emittype inapi-client/src/posthog-client.ts, and the stories. - The desktop app ships on its own cadence, so an older build will keep sending
emitin its PATCH body for a while. DRF ignores unknown fields, so those writes become no-ops and the scout stays live. That is the right way for it to fail, but confirm it rather than assume it.
Analytics: setting on the desktop config-changed event drops an enum member, and the detail-viewed event drops a property on both clients. Historical events stay readable, so this is deletion, not a rename.
Docs and generated files: the run-posture sections in authoring-scouts and exploring-scouts, the harness CLAUDE.md, the field mentions in products/signals/mcp/tools.yaml, and a hogli build:openapi run for the generated TypeScript.
Do not remove these
emit is overloaded in this codebase, and three neighbors read like the same thing:
- The other three preflight gates:
scout_config_missing,ai_processing_not_approved,source_disabled. These are the fail-closed path and the org consent gate._preflight_emit_gateskeeps its shape, minus one branch. can_emitinskill_loader.pyand the project profile. That is whether a skill was granted theemit_reporttool, which is a different mechanism.- The
scout-emit-signalandscout-emit-reportMCP tool names, and theemitted_count/emitted_finding_ids/emitted_report_idsrun tallies. Those name the act of emitting, not the setting.
Migration
The column is a plain unindexed boolean on a product table, with no constraint, no index, and no ClickHouse mirror. It is not on a hot table. Two phases, per /django-migrations:
- Remove every reference, then a state-only removal wrapped in
SeparateDatabaseAndState. Deploy. A DjangoSELECTlists every model field, so dropping the column while any running code still expects it breaks reads mid-deploy. DROP COLUMNin a later migration, once the first has been out for a full deploy cycle.
One decision for whoever picks this up
A handful of scouts in PostHog's own project are dry-run today, and they look like custom ones still being tuned. When the setting goes, they either start posting to the inbox or they stop.
Suggested default: disable them and tell their owners. From the inbox's point of view disabled and dry-run already look identical, disabled stops paying for discarded output, and the owner can turn it back on in one click. Flipping them live instead means someone gets output they never asked for from a scout they thought was still in staging. Worth a look at whether any enrolled team outside our own project is in the same position before choosing.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing SignalScoutConfig.emit from models.py through emit.py, structured_output.py, runner.py, serializers, views, and admin.py, then review the web and desktop files listed in the issue. Follow /django-migrations for the two-phase field removal and run hogli build:openapi. Done means the setting, controls, badges, analytics references, docs, and generated files are removed while real output gates remain intact and legacy desktop PATCHes are harmless.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python, react, typescript
- Domain
- backend, databases, desktop, documentation, frontend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100