redhat-et / redhat-et/ProtoBot

Clarify .skillsaw.yaml `content-paths` semantics to prevent recurrence of the AGENTS.md false-positive on PR #126

Open Beginner friendly
#136 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

documentation infra ready-for-triage triaged
Dominant language
Go
Stars
5
Forks
6
Avg merge
23h 24m
Merged PRs (30d)
66

Description

What happened

On PR #126, the author added two new hierarchy entries to AGENTS.md using markdown link syntax. Starting in round 1 (2026-09-16T14:13Z), fullsend-ai-review's style-conventions finding argued this was inconsistent formatting because 'AGENTS.md is not a Skillsaw content-path' — reasoning from .skillsaw.yaml's content-paths: list, which enumerates only docs/vision.md and docs/architecture.md. The agent repeated this unchanged finding across rounds 2, 3, and 4 despite the author's pushback. In round 5 (~16:32-16:54Z), the agent finally checked .skillsaw-baseline.json and discovered its premise was backwards: the checked-in baseline already records 13 baselined content-unlinked-internal-reference violations for AGENTS.md's existing plain-backtick references, and the skillsaw pre-commit hook (.pre-commit-config.yaml lines 64-67) runs with the tool's default pass_filenames: false, i.e. a whole-repo scan that ignores content-paths scoping entirely. Under this repo's strict: true, fail-on: info config, the author's original markdown-link choice was correct — it avoided introducing new, non-baselined violations that would fail CI — and the agent retracted the finding, explicitly stating the 'not a content-path' counter-argument used in rounds 1-4 'does not [hold], since the baseline proves the file is scanned regardless' (see PR #126, sticky comment https://github.com/redhat-et/ProtoBot/pull/126#issuecomment-5698922896, round-5 entry).

What could go better

I'm highly confident this cost 4 wasted review rounds on a finding that was wrong from round 1, because the review agent (and, by the same misreading, a human skimming the config) could reasonably conclude content-paths scopes the whole hook's file coverage — it does not, and nothing in .skillsaw.yaml says so. I'm less certain whether this is best fixed by clarifying the config (this proposal) versus changing review-agent behavior (see the companion proposal to the agents repo) — both are worth doing, and they're complementary: the config comment prevents the specific confusion that occurred here; the agent-behavior change generalizes to other lint tools with similarly non-obvious scoping.

Proposed change

Add an inline comment above content-paths: in .skillsaw.yaml stating that this list does not restrict which files the skillsaw pre-commit hook scans (the hook runs whole-repo via the tool's default pass_filenames: false), and that .skillsaw-baseline.json — not content-paths — is the source of truth for whether a given file/pattern is already exempted from a rule. This is a one-line comment change in a config file already tracked in this repo.

Validation criteria

Re-run this exact review scenario (a markdown-link vs. plain-backtick formatting question about a file outside content-paths) on the next PR that touches AGENTS.md or another non-content-paths file with existing baselined Skillsaw violations. The review agent (or a human reviewer reading the comment) should reference .skillsaw-baseline.json directly rather than reasoning from content-paths scope, and should not take more than 1 round to reach the correct conclusion about whether new formatting introduces a non-baselined violation.


Generated by retro agent from https://github.com/redhat-et/ProtoBot/pull/126

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read .skillsaw.yaml alongside .pre-commit-config.yaml and .skillsaw-baseline.json to confirm how content-paths, whole-repository scanning, and exemptions relate. Add the requested explanatory comment above content-paths, then verify the comment accurately reflects the hook configuration and baseline behavior described in PR #126.

Written by the indexing model from the issue text.

Assessment

Tech stack
yaml
Domain
tooling
Issue type
Documentation
Difficulty
1/5
Estimated time
Under an hour
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.