pingdotgg / pingdotgg/t3code

Verifier stops at the first issue: verify prompt should require a complete check pass with all findings

Open
#7,780 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
23k
Forks
5.9k
Avg merge
11h 14m
Merged PRs (30d)
357

Description

Symptom

The verifier ends its turn at the first problem it finds instead of completing its check plan and reporting everything, costing a full work + verify round trip per issue.

Root cause (verified)

The verify role's own prompt encourages early exit — STAGE_OWNERSHIP_REQUIREMENTS (apps/server/src/orchestration/stageResolution.ts:100-103):

"if code needs repair, report the exact failure so the PM can return it to a work stage."

Singular "failure", explicit exit ramp, no instruction to finish the remaining checks or enumerate all findings. The PM-side prompts reinforce one-defect-per-round framing:

  • pmSystemPrompt() (Layers/PmRuntime.ts:193,196): "if verification finds a code defect…"
  • Built-in playbook (builtInPlaybooks.ts:31-33): "If verify finds problems, send the findings to Work…"

There are no turn caps cutting verify short (verified: no maxTurns anywhere); this is purely prompt-driven.

Fix plan

Reword the verify ownership requirements to require running the full planned check set and reporting all findings as an enumerated list before ending the turn. Align the verify bullets in pmSystemPrompt, the built-in playbook, and the handoffWorker tool description (pmTools.ts:909) with the same framing.

Files: stageResolution.ts, Layers/PmRuntime.ts, builtInPlaybooks.ts, pm/pmTools.ts. Tests: stageResolution.test.ts:64 (asserts current verify prefix text), builtInPlaybooks.test.ts.

Note: structured findings artifacts (machine-readable issue lists accumulated across verify attempts) are a deliberate non-goal for this ticket — prompt fix only.

Acceptance criteria

  • Verify prompt instructs a complete pass + enumerated findings before turn end.
  • PM system prompt/playbook/handoffWorker description no longer frame verification defect-by-defect.
  • Prompt unit tests updated.

Size: S

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with STAGE_OWNERSHIP_REQUIREMENTS in apps/server/src/orchestration/stageResolution.ts and compare the verification wording in Layers/PmRuntime.ts, builtInPlaybooks.ts, and pm/pmTools.ts. Run the prompt tests at stageResolution.test.ts:64 and builtInPlaybooks.test.ts. Done means the verify instructions require a complete check pass with all findings enumerated, with matching wording across the PM prompts and handoffWorker description.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
developer-experience
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.