Comfy-Org / Comfy-Org/comfy-multi-player

Regression gap: no test pins the CodeRabbit CI-contract instruction against the actual CI gate list (#80 follow-up)

Open
#94 0 comments 0 reactions 1 assignee Claimed by @christian-byrne View on GitHub
Dominant language
TypeScript
Stars
5
Forks
1
Avg merge
12h 22m
Merged PRs (30d)
121

Description

## Gap

The #80 fix (`bb8c4c0f`) audited the CodeRabbit `path_instructions` blocks and corrected the CI-contract instruction so it names the active required CI gates. But no test on `main` compares the instruction's gate list against `.github/workflows/ci.yml` — so a future edit that removes a gate from the instruction (or adds a CI gate without updating the instruction) passes all current tests.

## Evidence (origin/main @ f8c1638e)

- Existing generator and claim-marker tests verify copying/substring presence, not cross-file consistency with the workflow definition.
- Concretely: deleting `check:imports` from the CI-contract instruction leaves the suite green today.
- Searched `test/`, `.agents/checks/`, `scripts/` for each CI gate name + `path_instructions` — no test derives the expected list from `ci.yml`.

## Missing test

Parse the required CI steps out of `.github/workflows/ci.yml` (job steps / required checks) and assert the generated CodeRabbit CI-contract instruction names every one. This pins the contract the same way the claim-marker tests pin wording.

## Context

- Found during closed-bug regression audit (OPP-33): all 14 other audited closed bug fixes have specific regression coverage; this was the only gap.
- Risk: Medium — silent divergence between CI reality and the reviewer contract, the exact failure mode #80 fixed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.