Comfy-Org / Comfy-Org/ComfyUI_frontend

.agents/checks: 22 of 24 review profiles are invoked by nothing; 0 of 24 gate a merge, and the dead import-graph check is the one the ECS branch needs

Open
#15,636 0 comments 0 reactions 1 assignee Claimed by @christian-byrne View on GitHub
area:CI/CD developer experience documentation
Dominant language
TypeScript
Stars
2k
Forks
699
Avg merge
1d 7h
Merged PRs (30d)
490

Description

## Summary

`.agents/checks/` holds **24 review-check profiles**. Nothing in the repo invokes 22 of them. The two exceptions are loaded by CodeRabbit, which comments but does not block. **Zero of the 24 gate a merge.**

Three of the 24 also name tools that are not installed anywhere in the repo, so they could not run even if something called them.

This is not a claim that the checks are bad — several are good rulebooks. It is a claim that the directory currently reads as coverage and delivers none, and that the highest-value one is dead in a way that is actively costing the ECS migration.

Measured at `origin/feature/ecs-migration` @ `62c36b3e64` (merge base `a2603c59a6` = `origin/main`). Same results on `main` for everything except where noted.

## Evidence

**Denominator**: `git ls-tree -r --name-only 62c36b3e64 .agents/checks/` → 25 files: 24 `.md` profiles + `eslint.strict.config.js`.

Invocation surfaces, all checked at that ref:

| Surface | Result | Control |
| --- | --- | --- |
| `.github/workflows/` (70 files) | **0** files contain `agents/checks` | 66 of the same 70 contain `runs-on`, so the loop read them |
| `package.json` `scripts` (74 scripts) | **0** references | — |
| `.husky/pre-commit` | `lint-staged` + `check-unused-i18n-keys.ts` — 0 references | — |
| `.husky/pre-push` | `pnpm knip` — 0 references | — |
| `lint-staged.config.ts` | oxfmt / oxlint / eslint / stylelint / vue-tsc — 0 references | — |
| whole tree, per-file basename grep | 3 of 24 referenced outside `.agents/checks/` | control pattern `import-graph` returns its own file |

Per-check result:

| Referenced by | Checks |
| --- | --- |
| CodeRabbit (`.coderabbit.yaml`), advisory | `test-quality.md` (lines 119, 124, 153), `playwright-e2e.md` (line 158) |
| Prose only (a human or agent must choose to read it) | `adr-compliance.md` (`AGENTS.md:192`), `playwright-e2e.md` (`browser_tests/AGENTS.md:11`, `browser_tests/README.md:19`) |
| **Nothing, anywhere** (21) | `accessibility`, `api-contract`, `architecture-reviewer`, `bug-hunter`, `coderabbit`, `complexity`, `ddd-structure`, `dep-secrets-scan`, `doc-freshness`, `dx-readability`, `ecosystem-compat`, `error-handling`, `import-graph`, `memory-leak`, `pattern-compliance`, `performance-profiler`, `regression-risk`, `security-auditor`, `semgrep-sast`, `sonarjs-lint`, `vue-patterns` |

CodeRabbit is live — 32 of the 100 most recent issue comments on this repo since 2026-08-19 are `coderabbitai[bot]` — but it is not a required status check. Ruleset `ProtectMain` (id 991238, active, `~DEFAULT_BRANCH`) requires exactly `test`, `lint-and-format`, `e2e-status`, `website-e2e`.

**Missing tools.** From `package.json` at the same ref:

```
eslint-plugin-sonarjs -> ABSENT (required by .agents/checks/sonarjs-lint.md and by
.agents/checks/eslint.strict.config.js, which imports it)
dependency-cruiser -> ABSENT (required by .agents/checks/import-graph.md)
semgrep -> ABSENT (required by .agents/checks/semgrep-sast.md)
```

`knip.config.ts:71` ignores `.agents/checks/eslint.strict.config.js` with the comment "Agent review check config, not part of the build" — knip is the only tool in the repo that knows the file exists.

## Why `import-graph.md` specifically is the expensive one

It is the only profile in the set that would enforce import direction, and it is dead three times over:

1. Nothing invokes it.
2. Step 3 branches on a root `.dependency-cruiser.js` / `.cjs`. `git ls-tree -r --name-only 62c36b3e64 | grep -i dependency-cruiser` → **nothing**. It therefore always takes the `--no-config` path, whose built-in ruleset is `no-circular`, `no-orphans`, `not-to-dev-dep`, `no-duplicate-dep-types` — **no layer-direction rule**. The profile's own "What It Catches" table lists "Custom layer rules" as the row that matters, and that row depends on the config that does not exist.
3. `dependency-cruiser` is not a dependency. (Step 1 uses `pnpm dlx`, which would fetch it, so this is the least load-bearing of the three.)

Meanwhile the thing it would have caught is happening. `src/lib/litegraph` imports of `@/stores` went **14 → 40 import lines** (4 → 11 non-test files) between `a2603c59a6` and `62c36b3e64`; `@/renderer` went 26 → 35. eslint's `import-x/no-restricted-paths` does not cover `src/lib/**` (see #14871, #14872, #15583), oxlint has no layer rule, and knip does not model direction. **Nothing automated catches it.** Verified by execution: probe files placed in a private worktree at `62c36b3e64` importing `@/renderer` from `src/lib/litegraph/src/` produce **0 eslint messages**, while a `var` probe in the same directory produces `no-var` at error — so the zero is "rule not applied", not "file not linted".

## Suggested fix

Decide per check, not for the directory as a whole. Three viable answers and one that is not viable:

- **Wire it.** Anything with a deterministic runner belongs in CI or in `lint-staged`. `import-graph` is the strongest candidate: add `dependency-cruiser` and a `.dependency-cruiser.js` that encodes the layer hierarchy, or — cheaper and already half-written — merge **#14872** and extend `import-x/no-restricted-paths` instead, and delete `import-graph.md`.
- **Declare it.** For the LLM-reviewer profiles, add a `path_instructions` entry in `.coderabbit.yaml` the way `test-quality.md` and `playwright-e2e.md` already have. That is a real consumer and it costs one YAML line.
- **Delete it.** A profile with no runner and no reader is a claim nobody tests.
- **Not viable: leave it.** The directory is cited by `AGENTS.md:192` as "automated validation rules". It is read by agents as evidence that a rule is enforced.

Related: #15585 (the ADR-compliance automation is scoped to `src/ecs/`, which never existed), #15583 (`src/world/` zone guards a deleted directory), #14871 / #14872 (the `src/lib` layer zone), #15552 (agent-facing docs citing things that no longer exist). Same class: agent-facing guidance that describes enforcement which is not wired up.

## How this was checked

All counts above are from `git ls-tree` / `git grep` / `git show` at a pinned ref, and every negative is paired with a control that returns non-zero on the same instrument. eslint results are from `ESLint#calculateConfigForFile` and from a real `eslint --format json` run on files on disk in a private worktree; `lintText` with a synthetic path was tried first, returned a parse error and zero findings **on every arm including the known-good control**, and was discarded as a zero-vs-zero measurement.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.