norrisaftcc / norrisaftcc/algorithm-shodann

[FEATURE] Let the agent fleet gate a review, starting with Citizen Zero on 'what would you do next'

Open
#52 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component: persona educational-priority priority: 1 status: needs-review type: feature
Dominant language
Python
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

The fleet currently reports to the maintainer. citizen-zero in particular produces exactly the artefact a review needs — a beginner's account of what a review left them able to do — and it produces it after the review has already been posted to a human.

The proposal

Run citizen-zero against the composed comment before it is posted, and treat "What I would do next: nothing" as a blocking finding.

That is the one failure mode the existing checkers structurally cannot see. validator.py knows the shape a response must take. groundedness.py knows whether it invented a metric. Neither can tell whether the result leaves a student able to act, because both have seen the prompt.

Why this is not just another validator

.claude/agents/README.md § The three positions sets this out: oracle-warden asks does it run, clive-prompt-warden asks does it comply, citizen-zero asks does it teach — and is the only one that cannot cheat, because its tool list is empty and it has never seen the code.

Precedent that it works: Citizen Zero caught a review promising readings "measured, not interpreted" and then opening "The Algorithm is deeply pleased", and separately caught a first submission being told it compared against a predecessor that did not exist. Both are recorded in design_docs/EARLY_RUNS.md. No mechanical checker found either.

What needs deciding

  1. Where it runs. In the review pipeline (latency, cost, and a second model call on every PR), or as a maintainer-invoked gate on template changes and new modes? The second is cheap and probably where this starts.
  2. What is blocking. "No action and no question" is the documented failure (.claude/agents/citizen-zero.md § What you report). "I would ask my instructor about X" is explicitly a success — a real thing the student cannot resolve alone, named well enough to take to office hours. Do not let an implementation treat those as the same outcome.
  3. Persona variance. Citizen Zero rolls a thin persona per run, deliberately, so two runs of the same review do not agree. A gate built on it needs several runs or a tolerance, not a single verdict.
  4. Whether the report reaches the citizen. Probably never — it is testimony about them, and PRD.md §7 puts surveillance students cannot see permanently out of scope. Worth stating explicitly so nobody builds it the other way.

Relevant documentation

What Where
The three epistemic positions, retargeting rules .claude/agents/README.md
What Citizen Zero reports and what counts as failure .claude/agents/citizen-zero.md
Findings the mechanical checkers missed design_docs/EARLY_RUNS.md
Shape and vocabulary checks this does not duplicate src/shodann/validator.py, src/shodann/groundedness.py
Surveillance boundary PRD.md §7

Depends on #28 in spirit — the fleet should be exercised before it is wired into a pipeline.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading .claude/agents/README.md, .claude/agents/citizen-zero.md, design_docs/EARLY_RUNS.md, and PRD.md §7, then compare the existing checks in src/shodann/validator.py and src/shodann/groundedness.py. The implementation needs an agreed pipeline entry point and explicit decisions about blocking findings, repeated runs, and report visibility. Done means the gate's scope and behavior are documented and exercised before pipeline integration.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.