addyosmani / addyosmani/agent-skills

Resolve skill-validator heuristic limitations before the PR-B test battery locks them in

Open
#387 5 comments 2 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
94.1k
Forks
10k
Avg merge
3d 15h
Merged PRs (30d)
23

Description

Surfaced during review of #379. These are pre-existing behaviors of `scripts/lib/skill-lint.js`, not regressions from the extraction, but they should be triaged as bug-vs-intended before the PR-B test battery codifies the current heuristics as the contract:

- **Frontmatter parsing** is a YAML-like subset: it accepts malformed YAML and non-string descriptions, rejects valid folded/commented YAML, and mismeasures the real length of multiline descriptions.
- **Section checks** use substring matching, so headings inside fenced code blocks or `###` sub-headings can satisfy a required-section rule. Contract question to resolve: `docs/skill-anatomy.md` describes the sections as recommended and allows equivalent headings, while the validator treats the exact headings as required.
- **Cross-reference patterns** miss some reference styles already used in the corpus and can match references inside fenced examples.
- **Trigger check** is satisfied by a negative phrase such as "Do not use when...".

For each: decide whether the current behavior is the intended contract (and document it) or a bug to fix, so PR-B tests assert the decided behavior rather than freezing today's heuristics. Credit to @federicobartoli for surfacing these while reviewing #379.

Contributor guide

Open the contributing guide

Research direction

Start in `scripts/lib/skill-lint.js`, where each questionable heuristic is implemented. Then check `docs/skill-anatomy.md` to compare documented intent for section requirements and references. Inspect the PR-B test battery to see current assertions, then add/update tests for the four scenarios and align docs so each item is marked as intended behavior or a bug. It is done when PR-B enforces an explicit contract rather than today’s accidental heuristics.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
documentation, testing-qa, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
63/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.