redhat-et / redhat-et/ProtoBot
Clarify .skillsaw.yaml `content-paths` semantics to prevent recurrence of the AGENTS.md false-positive on PR #126
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 5
- Forks
- 6
- Avg merge
- 23h 24m
- Merged PRs (30d)
- 66
Description
What happened
On PR #126, the author added two new hierarchy entries to AGENTS.md using markdown link syntax. Starting in round 1 (2026-09-16T14:13Z), fullsend-ai-review's style-conventions finding argued this was inconsistent formatting because 'AGENTS.md is not a Skillsaw content-path' — reasoning from .skillsaw.yaml's content-paths: list, which enumerates only docs/vision.md and docs/architecture.md. The agent repeated this unchanged finding across rounds 2, 3, and 4 despite the author's pushback. In round 5 (~16:32-16:54Z), the agent finally checked .skillsaw-baseline.json and discovered its premise was backwards: the checked-in baseline already records 13 baselined content-unlinked-internal-reference violations for AGENTS.md's existing plain-backtick references, and the skillsaw pre-commit hook (.pre-commit-config.yaml lines 64-67) runs with the tool's default pass_filenames: false, i.e. a whole-repo scan that ignores content-paths scoping entirely. Under this repo's strict: true, fail-on: info config, the author's original markdown-link choice was correct — it avoided introducing new, non-baselined violations that would fail CI — and the agent retracted the finding, explicitly stating the 'not a content-path' counter-argument used in rounds 1-4 'does not [hold], since the baseline proves the file is scanned regardless' (see PR #126, sticky comment https://github.com/redhat-et/ProtoBot/pull/126#issuecomment-5698922896, round-5 entry).
What could go better
I'm highly confident this cost 4 wasted review rounds on a finding that was wrong from round 1, because the review agent (and, by the same misreading, a human skimming the config) could reasonably conclude content-paths scopes the whole hook's file coverage — it does not, and nothing in .skillsaw.yaml says so. I'm less certain whether this is best fixed by clarifying the config (this proposal) versus changing review-agent behavior (see the companion proposal to the agents repo) — both are worth doing, and they're complementary: the config comment prevents the specific confusion that occurred here; the agent-behavior change generalizes to other lint tools with similarly non-obvious scoping.
Proposed change
Add an inline comment above content-paths: in .skillsaw.yaml stating that this list does not restrict which files the skillsaw pre-commit hook scans (the hook runs whole-repo via the tool's default pass_filenames: false), and that .skillsaw-baseline.json — not content-paths — is the source of truth for whether a given file/pattern is already exempted from a rule. This is a one-line comment change in a config file already tracked in this repo.
Validation criteria
Re-run this exact review scenario (a markdown-link vs. plain-backtick formatting question about a file outside content-paths) on the next PR that touches AGENTS.md or another non-content-paths file with existing baselined Skillsaw violations. The review agent (or a human reviewer reading the comment) should reference .skillsaw-baseline.json directly rather than reasoning from content-paths scope, and should not take more than 1 round to reach the correct conclusion about whether new formatting introduces a non-baselined violation.
Generated by retro agent from https://github.com/redhat-et/ProtoBot/pull/126
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read .skillsaw.yaml alongside .pre-commit-config.yaml and .skillsaw-baseline.json to confirm how content-paths, whole-repository scanning, and exemptions relate. Add the requested explanatory comment above content-paths, then verify the comment accurately reflects the hook configuration and baseline behavior described in PR #126.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- yaml
- Domain
- tooling
- Issue type
- Documentation
- Difficulty
- 1/5
- Estimated time
- Under an hour
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 88/100