spec-kitty / spec-kitty/spec-kitty

[process] acceptance records anchor commit SHAs that rebases orphan, twice on one PR

Open
#4,230 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.6k
Forks
165
Avg merge
13h 53m
Merged PRs (30d)
336

Description

_Originally filed in error as spec-kitty/spec-kitty-design#402 during epic #319 there; that repo is the design library and should never carry spec-kitty CLI issues. Re-filed here; the original is closed and points at this one._

Found by two lenses at the pre-merge gate, on both children of epic #319.

## The pattern

`kitty-specs//acceptance-matrix.json` and `issue-matrix.json` record evidence as commit SHAs and `git diff` ranges. On a branch that rebases, those SHAs are orphaned and the recorded proof becomes unreproducible — by anyone, permanently, including the auditor the record exists for.

Observed on PR #341:

- FR-019's evidence reads *"Re-verified against current head 94d2cdb4 … at head_sha=94d2cdb4…, the PR's exact current head."* `git merge-base --is-ancestor 94d2cdb4 HEAD` exits 1 — that commit is 95 commits behind HEAD and HEAD contains 84 it does not. The re-verification was correct when written and was invalidated by the next rebase ~20 minutes later.
- FR-002 / FR-011 / FR-018 and the `#320` issue-matrix row cite `git diff` ranges over four commits that are all orphans. Only one SHA in those ranges survives, and it is a train merge, not this branch.

The same shape appeared on PR #339 earlier in the same epic.

## Why chasing the SHA does not fix it

Each correction is itself a commit, which mints a new head, which invalidates the correction. On a train landing roughly every 25 minutes, the record can never catch up — this is structural, not a discipline failure.

## What survives a rebase

The *substance* was independently re-verified and holds in both cases: ratchet files byte-identical to the train, `tokens.css` untouched, baselines present on disk and still referenced at the cited line numbers, story ids present in the built index.

That is the clue. Content-addressed evidence survives a rebase; commit-addressed evidence does not.

## Suggested direction

Record acceptance evidence against things a rebase cannot move:

- file paths and line numbers,
- blob hashes (`git rev-parse HEAD:` — stable across rebases when content is unchanged),
- story ids,
- test names,
- CI **job conclusions keyed to the assertion**, rather than to a head SHA.

Where a SHA genuinely is the evidence (a CI run, for instance), pair it with the content hash of what it proved, so a later reader can confirm the artifact is unchanged even when the commit is gone.

Worth deciding centrally, since `acceptance-verdict` writes these records and could enforce the convention rather than leaving each mission to invent it.

Contributor guide

Open the contributing guide

Research direction

Start by locating the acceptance-verdict entry point that writes acceptance-matrix.json and issue-matrix.json, then inspect how evidence currently records commit SHAs and git diff ranges. Define the content-addressed evidence convention and the assertion-level CI linkage before changing the records. Done means acceptance evidence remains independently verifiable after a branch rebase.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, python
Domain
cli, devtools, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.