NVIDIA / NVIDIA/SkillSpector

AS3 flags current-skill SKILL.md self-references as Agent Snooping

Open
#500 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
17.9k
Forks
1.5k
Avg merge
5d 10h
Merged PRs (30d)
66

Description

Summary

SkillSpector 2.11.1 reports AS3 / Skill Enumeration when a skill document merely names its own manifest path, for example:

Root skill: `skills/example-skill/SKILL.md`

There is no filesystem read, enumeration, or access to another installed skill. A declarative self-reference is therefore classified as Agent Snooping and can fail downstream security gates.

Reproduction (v2.11.1)

Create example-skill/SKILL.md:

---
name: example-skill
description: Example skill.
---

# Team guidance

Root skill: `skills/example-skill/SKILL.md`

Run:

skillspector scan ./example-skill --no-llm --format json

Current result: an AS3 Skill Enumeration finding at MEDIUM severity and confidence 0.8 whose matched text is skills/example-skill/SKILL.md.

Expected result: no AS3 finding for this line because the path identifies the currently scanned skill, not a peer skill.

Root cause

The AS3 path pattern matches every token shaped like skills/<non-CURRENT-name>/SKILL.md, independent of whether <name> identifies the current skill:

r"skills?/(?:(?!CURRENT)[A-Z][A-Za-z0-9_-]+)/SKILL\.md"

The analyzer applies the pattern with re.IGNORECASE. It does not compare the captured path segment with the scan-root basename or manifest name.

Expected behavior / acceptance criteria

  • A current-skill reference such as Root skill: skills/example-skill/SKILL.md does not emit AS3.
  • Current-skill identity works from both the scan-root basename and manifest name, including temporary clone/extraction directory names.
  • References to a different skill remain AS3, including the existing open("skills/other-skill/SKILL.md").read() positive control.
  • Explicit enumeration of installed skill directories remains AS3.
  • AS1 and AS2 behavior remains unchanged.
  • Add paired negative and malicious positive regression tests.
  • Filtering occurs before inspection-ledger emission so completed ledger events do not retain stale emitted finding IDs.
  • Verify the static analyzer and a real no-LLM CLI scan of the minimal fixture.

Suggested direction

Capture the skill-name segment in the path-only AS3 match, derive normalized current-skill identifiers from the scan root and manifest, and suppress only AS3 findings whose path segment identifies the current skill. Preserve peer-skill path detection and the existing explicit read/enumeration detections.

Related

  • #75 and #96 introduced AS3 with the intended scope of skills/*/SKILL.md outside the scanned skill.
  • #326 is related report-surface work, but it does not fix this false positive.

AI assistance disclosure: Codex helped inspect the current source, reduce an observed CI finding to this public minimal reproduction, and draft the regression criteria. I reviewed the evidence and proposed scope.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the AS3 path-only match and the scan-root or manifest identity handling described in the issue. Add paired negative and malicious positive regression tests, then run the static analyzer and the minimal no-LLM CLI scan; done means current-skill references are suppressed while peer-skill and explicit enumeration findings remain, without stale ledger IDs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli, security, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.