Migrate owasp:llmNN probe tags to year-qualified owasp:YYYY:llmNN format
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9.3k
- Forks
- 1.3k
- Avg merge
- 5d 1h
- Merged PRs (30d)
- 14
Description
Spun out of #1919 and #1920 per @jmartin-tech's review.
Background
agent_breaker.AgentBreaker is currently the only probe tagged owasp:llm07 / owasp:llm08, while its doc_uri points at LLM06 (Excessive Agency) in the 2025 list. #1920 attempted to retag just that one probe, but as noted in review that would leave main inconsistent: the owasp:llm06 tag in garak was defined against the 2023-24 OWASP Top 10, tag-based grouping selects and groups by prefix, and changing a single probe conflates taxonomy versions.
Proposal
Migrate every owasp:llmNN tag in the probe catalog (78 entries) to a year-qualified form, e.g. for AgentBreaker:
owasp:2023:llm07
owasp:2023:llm08
owasp:2025:llm06
owasp:2026:llm03
- At minimum, add the
2023year marker to all existing entries so their meaning is unchanged. - Add newer-list tags (
2025,2026) where a probe maps onto them. - Update tag selection and grouping logic that keys on the
owasp:llmprefix soowasp:2023:*,owasp:2025:*etc. group correctly. - Update docs that reference the prefix scheme.
Open questions
- Preferred spelling for the year-qualified prefix (
owasp:2023:llm07vs. an alternative). - Whether newer-list tags (
2025/2026) land in the same PR or a follow-up.
Happy to own this PR if the team wants to proceed.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the probe catalog entries described here, then trace the tag selection and grouping logic that currently keys on the owasp:llm prefix. Check the context in issues #1919 and #1920 before choosing the year-prefix spelling and scope for newer-list tags. Done means existing tags are year-qualified, applicable newer tags are handled, grouping remains correct, and referenced documentation is updated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, security, tooling
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100