microsoft / microsoft/PyRIT

FEAT: Add ANSI escape output scorers and Garak AnsiEscape scenario

Open
#2,683 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.5k
Forks
893
Avg merge
3d 50m
Merged PRs (30d)
165

Description

### Part of #511

`AnsiAttackConverter` (#577 → #597) covers the *prompt* side of garak's `ansiescape` probe. The other half discussed on #577 — a scorer that checks whether the model actually emitted terminal control sequences — was the first item under "Next Steps" in #597 but was never followed up. There is also no scenario for garak's two active probes, `ansiescape.AnsiRaw` (tier `OF_CONCERN`) and `ansiescape.AnsiEscaped` (`INFORMATIONAL`).

Right now PyRIT can send ANSI prompts but can't tell whether the target produced something a terminal, log viewer or renderer downstream would act on.

#### Proposed scope

Two PRs, scorers first since they're useful on their own:

1. **Scorers** in `pyrit/score/true_false/regex/`, following `XSSOutputScorer`:
- `AnsiEscapeOutputScorer` — raw sequences: `ESC [` (CSI), `ESC ]` (OSC) and the C1 code points U+009B / U+009D. Parity with garak's `detectors.ansiescape.Raw`.
- `EscapedAnsiOutputScorer` — escaped text forms that go live once something unescapes them: `\x1b`, `\033`, `\u001b`, `\27`, `\e` followed by `[` or `]`, plus `\x9b` / `\x9d`. Parity with `detectors.ansiescape.Escaped`.

Two classes rather than one with a mode, so raw and escaped findings stay distinguishable in memory and metrics — same split garak makes with its tiers.

2. **Scenario** `garak.AnsiEscape` with `Raw` / `Escaped` techniques on `PromptSendingAttack`, prompts from a local `garak_ansi_escape` dataset (garak's asks × tasks, repeat stubs, unescape stubs), mirroring `WebInjection` / `Doctor`. Plus exports, tests, the synchronized scanner docs and `third_party/garak-provenance.json` entries.

`AnsiAttackConverter` and the Foundry `ansi_attack` technique stay as they are.

#### One upstream detail

garak's `RAW_COMMANDS` contains `"\27"`, which in a Python string literal is octal U+0017, not ESC, so that entry can never match what it's meant to. I'd leave it out of the raw scorer. The decimal form is still covered on the escaped side, where `\27[` is literal text.

#### One question before I start

Is the two-PR split fine, or would you rather see scorers and scenario together the way #2064 shipped `PolicyPuppetryConverter` with `Doctor`?

#### References
- Parent: #511; converter: #577, #597
- Upstream (pinned to the garak revision in `third_party/garak-provenance.json`): [`probes/ansiescape.py`](https://github.com/NVIDIA/garak/blob/3f50ea5ff9cd7050099940647c15c39b07a93392/garak/probes/ansiescape.py), [`detectors/ansiescape.py`](https://github.com/NVIDIA/garak/blob/3f50ea5ff9cd7050099940647c15c39b07a93392/garak/detectors/ansiescape.py), [`resources/ansi.py`](https://github.com/NVIDIA/garak/blob/3f50ea5ff9cd7050099940647c15c39b07a93392/garak/resources/ansi.py)
- [LLM Output Can Take Over Your Computer](https://interhumanagreement.substack.com/p/llm-output-can-take-over-your-computer)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in pyrit/score/true_false/regex/ by reading XSSOutputScorer and compare the pinned garak ansiescape detectors and resources referenced in the issue. The scorer work is complete when raw and escaped ANSI findings are distinguishable and covered by tests; the broader scope also includes the AnsiEscape scenario, exports, scanner documentation, dataset, and provenance entries.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.