NVIDIA/garak

View on GitHub

the LLM vulnerability scanner

Stars: 9.3k
Forks: 1.3k
Open beginner issues: 10
Indexed issues: 258
Avg merge: 5d 1h
Merged PRs (30d): 14

Dominant language: Python
License: Apache-2.0
Last GitHub push: Sep 16, 2026
Latest indexed: Sep 19, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue

Beginner friendly Everything

Open All

258 open issues indexed

English only detectors misjudge targets tht reply in another language in both diirections Open

bug needs-triage

Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA/garak#2217 · Opened Sep 19, 2026
Digest of an aggregated report scores each probe/detector pairing from one contributing run only Open

needs-triage

Difficulty 4/5 3-5 days Newbie friendliness 72/100

NVIDIA/garak#2213 · 3 comments · Opened Sep 18, 2026
aggregate_reports drops payload_init and tree_data rows, so aggregated reports lose payload provenance and tree search history Open

needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 82/100

NVIDIA/garak#2211 · Opened Sep 18, 2026
analyze_log and qual_review crash on unscoreable (null) detector scores in attempt records Open

needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 20/100

NVIDIA/garak#2209 · Opened Sep 18, 2026
`MitigationBypass` and `Prefixes` use substring-anywhere matching, so a compliant response that names the topic scores as a refusal Open

needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 74/100

NVIDIA/garak#2197 · 2 comments · Opened Sep 16, 2026
detectors.apikey.ApiKey hangs during detector phase with parallel_attempts=32 (CPU freezes, 61GB RSS) Open

needs-triage

Difficulty 4/5 3-5 days Newbie friendliness 48/100

NVIDIA/garak#2189 · 1 comment · Opened Sep 14, 2026
Embodied probes: would garak take a detector that needs a simulator, or is that a separate tool? Open

for-maintainers needs-triage question

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/garak#2186 · Opened Sep 12, 2026
add reusable support for intent-aware iterative probes Open

needs-triage

Difficulty 5/5 Over a week Newbie friendliness 45/100

NVIDIA/garak#2180 · 2 comments · Opened Sep 11, 2026
bug: IntentProbe generates duplicate prompts within the same intent Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

NVIDIA/garak#2179 · Opened Sep 11, 2026
probe: reasoning theft attack Open

new plugin probes

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/garak#2178 · 2 comments · Opened Sep 10, 2026
generator: factor temp, seed variation up from `nim` to `OpenAICompatible` Open

generators

Difficulty 3/5 1-2 days Newbie friendliness 68/100

NVIDIA/garak#2177 · Opened Sep 10, 2026
AgentBreakerResult accepts out-of-schema judge verdict values Open

needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 76/100

NVIDIA/garak#2174 · 5 comments · Opened Sep 10, 2026
No trajectory-aware detector exists — even GOAT, garak's most sophisticated multi-turn probe, scores only one turn per conversation Open

needs-triage

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/garak#2173 · 1 comment · Opened Sep 10, 2026
ApiKey detector safe token masks a separate credential match Open

needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 76/100

NVIDIA/garak#2172 · Opened Sep 10, 2026
Dated chat model names are routed to the legacy completions endpoint Open

needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 88/100

NVIDIA/garak#2169 · Opened Sep 9, 2026
generate() only type-checks _call_model output on 2 of its 4 paths Open

needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 52/100

NVIDIA/garak#2160 · Opened Sep 5, 2026
map euai: tags onto intents Open

needs-triage

Difficulty 5/5 Over a week Newbie friendliness 45/100

NVIDIA/garak#2157 · 4 comments · Opened Sep 4, 2026
Buff._derive_new_attempt shares notes and detector_results dicts between attempts, so sibling attempts report the last attempt's scores Open

bug

Difficulty 2/5 1-3 hours Newbie friendliness 86/100

NVIDIA/garak#2155 · 1 comment · 1 reaction · Opened Sep 4, 2026
StringDetector matchtype "word" cannot match terms with a non-word character at either edge — 14 shipped Surge terms are dead Open

needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

NVIDIA/garak#2143 · Opened Sep 3, 2026
Bound generator retries to prevent indefinite hangs Open

architecture bug needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 68/100

NVIDIA/garak#2141 · 4 comments · Opened Sep 2, 2026
Migrate owasp:llmNN probe tags to year-qualified owasp:YYYY:llmNN format Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA/garak#2139 · 2 comments · Opened Sep 1, 2026
judge detectors: unparseable judge verdict is scored as a successful attack Open

needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 68/100

NVIDIA/garak#2136 · 3 comments · Opened Aug 31, 2026
Documentation: REST generator configuration broken in v0.16 — CLI flags changed without docs update Open

documentation needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 68/100

NVIDIA/garak#2131 · 2 comments · Opened Aug 29, 2026
elicitation_injection — completion-elicited obfuscated commands (distinct from encoding?) Open

for-maintainers needs-triage question

Difficulty 4/5 3-5 days Newbie friendliness 52/100

NVIDIA/garak#2128 · Opened Aug 29, 2026
Tool Surface Enumeration: Shared Recon Phase for Tool-Aware Probes Open

architecture for-maintainers

NVIDIA/garak#2126 · 3 comments · 1 reaction · 1 assignee · Opened Aug 28, 2026
intentservice global state (`is_loaded`/`intents_active`) not reset between tests Open

bug needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 78/100

NVIDIA/garak#2124 · 1 comment · Opened Aug 27, 2026
feat(cli): add --list_intents to enumerate the intent typology Open

cli

Difficulty 3/5 1-2 days Newbie friendliness 72/100

NVIDIA/garak#2123 · 2 comments · Opened Aug 27, 2026
analyze: uncorrected ASR intervals are indistinguishable from corrected ones (116 of 122 detectors) Open

needs-triage

NVIDIA/garak#2117 · 1 comment · 1 assignee · Opened Aug 25, 2026
Optional EvalPort interop for garak attempts and detector results Open

needs-triage

Difficulty 5/5 Over a week Newbie friendliness 38/100

NVIDIA/garak#2110 · 4 comments · Opened Aug 22, 2026
Agent telemetry support Open

architecture for-maintainers

Difficulty 5/5 Over a week Newbie friendliness 25/100

NVIDIA/garak#2106 · 4 reactions · Opened Aug 21, 2026
Probe: negotiated authenticated encrypted channels Open

needs-triage

Difficulty 5/5 Over a week Newbie friendliness 30/100

NVIDIA/garak#2093 · 1 comment · Opened Aug 18, 2026
Probe: Inductive alphabet-substitution probe for planted secret leakage Open

needs-triage

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/garak#2092 · Opened Aug 18, 2026
feature: extend the intent tree from 3 to 4 levels for sub-cases without a shared detector prompt space Open

needs-triage

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/garak#2091 · 2 comments · Opened Aug 18, 2026
feat(buffs): Caesar cipher buff for shift-encoding probe prompts Open

needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 25/100

NVIDIA/garak#2088 · Opened Aug 18, 2026
outputs_for() lacks the prompt_for() fallback: detectors silently see empty outputs when reverse translation is unpopulated Open

needs-triage

Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA/garak#2078 · 2 comments · Opened Aug 17, 2026
Azure model mapping and OpenAI context tables point at retired models Open

needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 55/100

NVIDIA/garak#2068 · Opened Aug 15, 2026
mitigation.Prefixes matches anywhere in the response, so compliant answers score as refusals Open

needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 68/100

NVIDIA/garak#2066 · 1 comment · Opened Aug 15, 2026
CI calculation reseeds the global numpy RNG, so --seed makes later numpy draws repeat Open

bug needs-triage

NVIDIA/garak#2062 · 6 comments · 1 assignee · Opened Aug 14, 2026
feature: expand intent→detector mapping coverage Open

detectors dontautoclose

Difficulty 3/5 1-2 days Newbie friendliness 62/100

NVIDIA/garak#2059 · 1 comment · Opened Aug 13, 2026
Bootstrap CI degenerates to zero width at 0%/100% pass rates, so no interval is shown at all Open

bug needs-triage

Difficulty 4/5 3-5 days Newbie friendliness 64/100

NVIDIA/garak#2033 · 5 comments · Opened Aug 9, 2026
Clarify AttackRogueString semantics for quoted or transformed trigger strings Open

documentation needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

NVIDIA/garak#2028 · 3 comments · Opened Aug 7, 2026
Probe.probe() raises IndexError when the prompt set is empty (translation can produce an empty set) Open

bug needs-triage

Difficulty 2/5 1-3 hours Newbie friendliness 72/100

NVIDIA/garak#2026 · 1 comment · Opened Aug 7, 2026
A probe that yields zero attempts leaves no trace in report.jsonl, so "did not run" and "was never configured" are the same absence Open

needs-triage question

Difficulty 3/5 1-2 days Newbie friendliness 68/100

NVIDIA/garak#2021 · 4 comments · Opened Aug 5, 2026
reporting: record detector-scoped output representation in hitlog for translated runs Open

needs-triage

Difficulty 5/5 Over a week Newbie friendliness 35/100

NVIDIA/garak#2001 · Opened Jul 30, 2026
ansiescape: detector and payloads cover only CSI and OSC introducers Open

needs-triage

Difficulty 3/5 1-2 days Newbie friendliness 72/100

NVIDIA/garak#1977 · 3 comments · Opened Jul 20, 2026
cas: Techniques and Intents tab, By technique - detail level Grouped and detail level All leaves don't connect Open

reporting ux

Difficulty 3/5 1-2 days Newbie friendliness 65/100

NVIDIA/garak#1972 · 6 comments · Opened Jul 16, 2026
feature: First-class support for Tool messages Open

architecture for-maintainers

NVIDIA/garak#1969 · 6 comments · 2 reactions · 1 assignee · Opened Jul 15, 2026
probe: log injection / log forging (CWE-117, CWE-93) — model output forges log entries when written to log files Open

needs-triage

Difficulty 4/5 3-5 days Newbie friendliness 50/100

NVIDIA/garak#1957 · 2 comments · Opened Jul 14, 2026
probe: RAG / vector-store / agent-memory *sleeper* poisoning (retrieval-time latent injection) Open

needs-triage

NVIDIA/garak#1950 · 3 comments · 1 reaction · 2 assignees · Opened Jul 13, 2026
probe: SVG injection (model output hits a downstream rasterizer / XML parser... XXE + SSRF) Open

Difficulty 4/5 3-5 days Newbie friendliness 45/100

NVIDIA/garak#1949 · 4 comments · Opened Jul 12, 2026

NVIDIA/garak

Get new issues in your inbox