garrytan / garrytan/gstack

/ship and /review Plan Completion Audit never searches docs/designs/, so it audits PRs against an unrelated plan file

Open
#2,768 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133k
Forks
19.9k
Avg merge
18h 46m
Merged PRs (30d)
26

Description

## Summary

v1.65.0.0 moved design docs into the repo:

> **Design docs land in your repo** (#703, #2000): office-hours writes
> `docs/designs/.md` ... Plan reviews prefer the repo-local doc when both exist.

`/plan-ceo-review`, `/plan-eng-review` and `/plan-devex-review` were updated — they all
take the shared `{{DESIGN_DOC_DISCOVERY}}` block, which reads `docs/designs/`.

The **Plan Completion Audit** in `/ship` and `/review` was not. Its search list still
predates the change, so a repo-committed design is invisible to it, and it falls through
to an unrelated plan file in `~/.claude/plans` and audits the PR against that.

gstack 1.78.0.0.

## Evidence

`ship/sections/plan-completion.md:24-25` and `review/sections/plan-completion.md:18-19`
are byte-identical:

```bash
# Search common plan file locations (project designs first, then personal/local)
for PLAN_DIR in "$HOME/.gstack/projects/$_PLAN_SLUG" "$HOME/.claude/plans" "$HOME/.codex/plans" ".gstack/plans"; do
```

`docs/designs/` is on none of them — `grep -c 'docs/designs'` returns **0** in both
files — while the comment on the line above says "project designs first".

Meanwhile `scripts/resolvers/design-doc-discovery.ts` exists precisely to answer "which
design doc wins", and is wired into the three plan-review skills but not here.

## Consequence

With the repo design invisible, the loop reaches `~/.claude/plans` and runs:

```bash
PLAN=$(ls -t "$PLAN_DIR"/*.md | xargs grep -l "$BRANCH" | head -1)
[ -z "$PLAN" ] && PLAN=$(ls -t "$PLAN_DIR"/*.md | xargs grep -l "$REPO" | head -1)
[ -z "$PLAN" ] && PLAN=$(find "$PLAN_DIR" -name '*.md' -mmin -1440 ... | head -1)
```

`$BRANCH` is frequently `master` or `main`, which nearly every plan file mentions
somewhere, so the first grep alone will match an arbitrary recent plan from an unrelated
project. Observed in a repo whose approved design sits at
`docs/designs/.md`: the audit resolved to a `~/.claude/plans` file belonging to a
different feature, and never saw the real design.

The step-3 validation ("read the first 20 lines and verify it is relevant ... treat as
no plan file found") does catch this when the model applies it, so the usual outcome is a
**silently skipped audit** rather than a wrong one. That is a soft, model-judgment guard
in front of a deterministic discovery bug, and it fails open in the direction of "no
audit ran" — which the skill reports as normal.

## Compounding: the slug here disagrees with the rest of gstack

`$_PLAN_SLUG` is computed by an inline `sed` over the remote URL, not by
`bin/gstack-slug`. Where the two disagree, the first leg points at a directory gstack
itself never writes to, so even a `~/.gstack`-resident design is missed. Filed
separately as #2767 (sticky slug not inherited by linked worktrees) — the two bugs share
the same root: plan discovery reimplements slug resolution instead of reusing it.

## Suggested fix

1. Use the shared discovery block. `scripts/resolvers/design-doc-discovery.ts` already
exists as the single source of truth for this question, and its own header explains
why it was extracted: "Drift between copies meant plan reviews could disagree about
which design doc wins." The Plan Completion Audit is another such copy that drifted.
At minimum, add the repo leg (`DESIGN.md`, then `docs/designs/*.md`) to the search.
2. Replace the inline slug `sed` with `bin/gstack-slug` so plan discovery and the rest
of gstack agree on which store directory belongs to this project.

## Secondary, same theme

The resolver's own repo selector is:

```bash
_REPODOC=$(ls -t "$_REPOTOP"/docs/designs/*.md 2>/dev/null | head -1)
```

Newest file in the directory, not the topic the branch is about. A repo with more than
one design doc hands plan reviews whichever was touched last, regardless of relevance.
Harmless with a single design doc, wrong as soon as there are two.

Contributor guide

Open the contributing guide

Research direction

Start with ship/sections/plan-completion.md and review/sections/plan-completion.md, then read scripts/resolvers/design-doc-discovery.ts and bin/gstack-slug. Compare the audit search order with the shared discovery block and verify how the slug is resolved. Done means both audits find repo-local docs/designs files and use the same project slug and discovery behavior as plan reviews.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell, typescript
Domain
cli, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.