TimZander / TimZander/claude

New skill: verify-pr-resolution — validate PR feedback was addressed, then approve & surface the complete link

Open
#164 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
1
Avg merge
1d 3h
Merged PRs (30d)
7

Description

Summary

Reviewer-side counterpart to respond-to-pr-feedback (#163). When a PR author pushes updates in response to review comments, this skill checks every thread (the user's own and others'), independently validates that each "resolved" thread is actually addressed by the latest commits (not merely flagged resolved), closes the user's threads with an acceptance comment where confirmed, and — once all threads are resolved to satisfaction — votes approve and hands the user a link to complete/merge the PR. Every decision is shown with justification, and the user chooses.

Closes the review loop: deep-review (produce) → respond-to-pr-feedback (author addresses, #163) → verify-pr-resolution (reviewer validates + approves).

Source detection (GitHub or ADO)

Same dual-source pattern as start-work / craft-pr:

  • No arg → the PR for the current branch.
  • #N / N / PR URL → that PR.
  • GitHub via gh pr view / gh api; ADO via repo_list_pull_request_threads / repo_vote_pull_request (resolve the repo GUID first).

Workflow

  1. Resolve the target PR and fetch its latest state — all comment threads (the user's and others'), each thread's status, and the commits added since the comments were raised.
  2. For each thread, determine who raised it, its current status, and what change (if any) claims to address it.
  3. Independently validate each thread rather than trusting the "resolved" flag:
    • Fetch the most recent commit(s) and inspect the diff at the comment's anchor (and the actual fix site, which may differ from the anchor).
    • Classify each: ✅ Confirmed resolved (evidence: commit SHA + file:line), ⚠️ Marked resolved but not validated (no corresponding change found, or the change doesn't address the point), or ↪️ Deferred (answered with a tracked follow-up issue/story — validate the link exists).
  4. Present the per-thread assessment with justification (status, verdict, evidence) and a recommended action. The user can override any verdict, ask questions, or add context.
  5. On the user's go-ahead:
    • The user's own confirmed threads → close with an acceptance comment.
    • Not-validated threads → leave open and post a comment explaining what's still missing; offer the user choices (request changes, comment, accept anyway, or convert to a follow-up).
    • Others' threads → the skill cannot resolve them on another reviewer's behalf; it reports their state and whether they block approval.
  6. Approval gate: only when all threads are resolved to satisfaction (the user's confirmed + others' resolved + deferrals accepted) and the user confirms — vote Approve and surface a link for the user to complete/merge the PR. Never auto-complete.
  7. Offer to notify the author/reviewers of the approval.

Validation method

  • Default (cheap, no worktree): fetch the PR head and diff the comment's anchored lines across the commits added after the comment was raised.
  • When validation needs building/running (e.g. "does this actually fix the bug?"): use a worktree (reuse-aware, per #163's convention) and note what was run.
  • Deferral path: a thread answered "deferred to <issue/story>" is validated by confirming the follow-up exists and is linked — not by a code change.

Design considerations

  • Trust but verify. Never accept a "resolved" flag at face value — the author may mark resolved without a real fix. The independent commit check is the core value of the skill.
  • Distinguish the user's threads from others'. The skill closes/accepts only the user's own threads; for others' it assesses and reports but does not resolve on their behalf.
  • Approval and completion are the user's call. Present the vote and a complete link; never vote or merge without explicit confirmation (outward-facing, significant).
  • Show justification for every verdict — concrete evidence (commit SHA, file:line), not just a yes/no.
  • Platform vote/complete mechanics: GitHub approve review (+ merge link); ADO repo_vote_pull_request (+10) + complete link. Never push to main; never auto-complete.
  • Idempotency: don't re-comment threads already accepted in a prior run.
  • Plumbing reuse: source-detection, GUID-resolution, and worktree helpers from start-work / craft-pr / respond-to-pr-feedback.
  • Pairs with #163: this skill consumes what that one produces (acceptance replies, deferral links).

Acceptance criteria

  • New skill (suggested name verify-pr-resolution) registered in the marketplace and installable.
  • Detects the PR from the current branch, or #N / N / URL, on both GitHub and ADO.
  • Lists every thread (the user's and others') with current status.
  • Independently validates each resolved thread against the latest commit(s) — does not trust the resolved flag.
  • Classifies each thread (confirmed / not-validated / deferred) with evidence and justification.
  • Closes the user's own confirmed threads with an acceptance comment.
  • Leaves unvalidated threads open with an explanatory comment and offers the user choices.
  • Only when all threads pass and the user confirms: votes Approve and surfaces a complete/merge link (never auto-completes).
  • Offers to notify the author/reviewers.
  • Deterministic logic (fetch threads, diff-based validation, post comments, vote) lives in a tested script; verdict judgment stays agent-side. Smoke test included.

Out of scope

  • Auto-completing/merging the PR.
  • Resolving other reviewers' threads on their behalf.
  • Producing new review findings (that's deep-review).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the existing start-work, craft-pr, and respond-to-pr-feedback implementations, including their source-detection, GUID-resolution, and worktree helpers. Trace how skills are registered in the marketplace, then define the tested script and smoke test around the listed GitHub and ADO workflows. Done means the acceptance criteria pass without auto-completing or merging a PR.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, python
Domain
developer-experience, devtools, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.