TimZander / TimZander/claude

deep-review: delta re-review — report fixed / still-open / new against the last reviewed SHA

Open
#222 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
0
Forks
1
Avg merge
1d 3h
Merged PRs (30d)
7

Description

Part of the deep-review: delivery rework milestone. Design context: #217 §4, §5F.

Problem

A PR with seven review rounds is normal, and each round carries real latency: review posted → author reads it (ten minutes, or two days) → author pushes → back to the reviewer. A PR can take a week to land, and nearly all of that is wall-clock waiting.

Deep-review has no memory across rounds. Round 5 re-reviews the whole branch from scratch and re-reports everything it finds, so the author has to diff two prose reviews by eye to work out what they actually still need to fix. Worse, a fresh exhaustive pass on round 5 surfaces new 💡 suggestions — a naming nit raised on round 5 costs another day of wall clock and buys a better variable name.

The cost of a review is not tokens. It is round trips.

Proposal

1. Delta mode. When a PR has been reviewed before, compare against the previously reviewed SHA and report three groups:

  • Fixed — findings from the last round that no longer reproduce
  • Still open — findings from the last round that do
  • 🆕 New — findings introduced by the commits since

The round-N comment is short because it is a delta, not because anything was suppressed. The full analysis still happens. The author sees the list shrink each round, which is a visible ratchet toward merge.

2. Round-aware severity ratchet. After round 1, only regressions and blockers get posted; new non-blocking findings are recorded in the artifact but not raised in the thread. Exhaustiveness in round 1 is a feature — it front-loads everything into a single trip. Exhaustiveness in round 5 is a tax.

3. State travels with the PR. The previously reviewed SHA and the prior finding set live in the machine payload embedded in the posted review comment (#217 §5C). No external store, no database, and any agent that fetches the comment can reconstruct the history.

Open questions

  • Fingerprint quality decides whether this works. A missed match reports a still-open finding as new — annoying but survivable. A false match reports an unfixed finding as fixed, which is a correctness bug in the review itself. Which way should the matcher be biased? (Toward "report as new," I think — but that means the ratchet is noisier than it looks.)
  • Is "no longer reproduces" the same as "fixed"? A finding can vanish because the author deleted the code, refactored around it, or because this run simply didn't surface it. Do we need to re-verify each prior finding explicitly rather than inferring from absence?
  • Who decides "blocking" for the ratchet — severity, or the independent blocking flag from the structured-findings issue?
  • Does the ratchet ever reset? A force-push that rewrites the branch, or a merge of a moved base, arguably invalidates the prior round.
  • First-round detection. How does the skill know a PR was reviewed before if the previous review was posted by a different person's session? Reading it back off the PR handles that — but only if the comment is findable and the payload survived.

Depends on

The structured-findings issue (fingerprints and the persisted payload are prerequisites) and the PR-comment renderer (which owns the comment the state lives in) — both in this milestone.

Acceptance criteria (provisional — pending #217)

  1. Re-reviewing a previously reviewed PR produces fixed / still-open / new groupings rather than a full re-listing.
  2. The comparison SHA and prior finding set are recovered from the PR itself, with no external state.
  3. A finding that survives an unrelated edit elsewhere in the same file is reported as still-open, not as new.
  4. First-time reviews are unaffected and still run the full exhaustive pass.
  5. The full analysis is still performed every round — only what is delivered is filtered.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the design context in #217 §4 and §5F, then read the structured-findings and PR-comment renderer work identified as prerequisites. Define how prior review state is recovered and compared, including first reviews, force-pushes, and fingerprint behavior; done means the provisional acceptance criteria are met without external state.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.