P6 false positive on HTML Output Rules documentation heading
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 17.9k
- Forks
- 1.5k
- Avg merge
- 5d 10h
- Merged PRs (30d)
- 66
Description
P6 reports a harmless HTML report-formatting heading as a HIGH-severity Direct Prompt Extraction finding.
Reproduced on main at 69dcdfb74487d361ba4c811d088cfdea2ff3a9dc (2.11.2) with this SKILL.md:
---
name: report-example
description: Generate a local HTML report from user-provided data.
---
# Report Example
## HTML Output Rules (Offline-Safe)
- Include figure titles, axis labels, and legends.
- Keep the generated HTML usable without network access.
Run:
skillspector scan ./report-example --no-llm --format json --output report.json
The report contains P6 at SKILL.md:8, matched text Output Rules, severity HIGH, confidence 0.85. The scan completes successfully; this finding contributes a risk score of 21 (CAUTION).
The heading describes the generated report's formatting requirements. It does not request disclosure of the assistant's system prompt or hidden instructions. P6 currently allows both your and system to be absent, so it interprets the noun phrase as an imperative. The existing exception for exactly ## Output Rules (Both Modes) does not cover this heading.
Expected behavior:
- Recognize unambiguous report-formatting headings without a P6 finding.
- Preserve actual prompt/rule extraction instructions, including instructions in headings, code blocks, and adjacent text.
- Preserve finding locations, accounting, and JSON/Markdown/SARIF output correctness.
- Add paired benign and malicious regressions and verify the complete CLI scan, not only the regex.
Related: #37 covers documentation false positives broadly; #453 addresses a different nominal print rules phrase. This issue tracks the report-formatting heading case specifically.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the P6 scan path using the supplied SKILL.md example and skillspector scan ./report-example --no-llm --format json --output report.json. Compare the benign HTML Output Rules heading with the existing prompt-extraction exception and the related cases in #37 and #453. Done means paired benign and malicious regressions pass while finding locations, risk accounting, and JSON, Markdown, and SARIF output remain correct in the complete CLI scan.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- security, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100