amirbena / amirbena/code-review-skill

Define the citation-grounding verification model

Open
#348 0 comments 0 reactions 1 assignee Claimed by @amirbena View on GitHub
area:review-quality maintainer-led priority:P2 type:research
Dominant language
Python
Stars
7
Forks
2
Avg merge
27m
Merged PRs (30d)
188

Description

## Type

Research

## Area

Review Quality

## Priority

P2 — Medium

## Problem

`shared/policies/architectural-placement.md` and `shared/policies/review-scope.md` already require bounded caller/callee traversal with concrete evidence before a placement finding, and `shared/templates/finding.md` already carries `confidence` / `contextual evidence` / `runtime validation` provenance. None of it proves a citation was produced by actually inspecting the code — only that the reviewer asserts so. `#182` will add observational execution telemetry (files inspected, symbols expanded), but there is no defined mapping from "a finding's `location`" to "was this inside what telemetry recorded as inspected." Without that mapping, telemetry can never be turned into a grounding signal by any future consumer, even a purely observational one.

## Goal

Produce a design record defining, as a specification only, what it means for a finding's citation to be "grounded" in `#182`'s recorded execution telemetry — consumable by a future citation-grounding cross-check, without deciding whether grounding should ever gate anything.

## Scope

- Define the correspondence rule between a finding's `location` / `evidence location` (`shared/templates/finding.md`) and `#182`'s telemetry fields (inspected files, expanded symbols, expansion rings).
- Cover the consolidated/root-cause finding case (`affected locations`, multiple sites) explicitly, not only single-site findings.
- Define the classification: grounded / ungrounded / indeterminate (e.g. telemetry missing or partial for that review).
- State explicitly that grounding is diagnostic-only and does not by itself imply any enforcement action — enforcement is a separate, later question.
- Note where the model's assumptions would break if `#182`'s telemetry schema changes shape.

## Non-Goals

- No implementation — no code reading real telemetry or real findings.
- Does not decide whether grounding should ever gate confidence/severity/decision.
- Does not modify `#182`'s own scope, schema, or observational boundary.

## Acceptance Criteria

- [ ] A design record defines the grounding correspondence rule.
- [ ] The consolidated-finding case is covered.
- [ ] The document states explicitly that the model is observational/diagnostic only, distinguishing it from `shared/policies/review-stopping-criteria.md`'s decision-affecting coverage concept.
- [ ] The model is reviewed against `#182`'s current telemetry field list for feasibility.

## Dependencies

Depends on: none (informed by, not blocked on, #182's implementation)
Blocks: a future citation-grounding cross-check (not yet filed — see the canonical design, below)
Relates: #182

## Validation

- Design record reviewed and merged.
- No code changes to validate.

## Canonical design

Part of the architecture defined in
[`docs/benchmark-measurement-architecture/benchmark-measurement-architecture-model.md`](docs/benchmark-measurement-architecture/benchmark-measurement-architecture-model.md),
§12.2 ("Provenance / citation-grounding verification"). Read the canonical
design before implementation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.