microsoft / microsoft/agent-governance-toolkit

conversation_guardian: invisible characters inside a keyword bypass all detection in normalize_text

Open
#3,500 3 comments 0 reactions 0 assignees View on GitHub
needs-review:HIGH
Dominant language
Python
Stars
6.3k
Forks
1.1k
Avg merge
5d 11h
Merged PRs (30d)
142

Description

### Summary

`normalize_text` in `agent-os/src/agent_os/integrations/conversation_guardian.py` strips five zero-width code points. Any other invisible character placed *inside* a keyword splits the word for every ``-anchored pattern in the module, so detection is bypassed completely.

### Reproduction

```python
from agent_os.integrations.conversation_guardian import (
ConversationGuardian, EscalationClassifier, OffensiveIntentDetector,
)

SHY = "­" # soft hyphen -- renders as nothing mid-word

clean = "urgent you must comply immediately no excuses"
evaded = clean.replace("urgent", "ur" + SHY + "gent") .replace("must", "mu" + SHY + "st") .replace("immediately", "imme" + SHY + "diately") .replace("excuses", "excu" + SHY + "ses")

c = EscalationClassifier()
print(c.score_message(clean)[0]) # 0.80 -- over the 0.6 threshold
print(c.score_message(evaded)[0]) # 0.00 -- no alert at all
```

End to end:

```python
words = ["exfiltrate", "the", "data", "escalate", "privileges", "impersonate"]
content = " ".join(w[:len(w)//2] + SHY + w[len(w)//2:] for w in words)
alert = ConversationGuardian().analyze_message("c1", "a", "b", content)
print(alert.severity, alert.action)
```

| message | `EscalationClassifier` | `OffensiveIntentDetector` | `analyze_message` |
| --- | --- | --- | --- |
| clean | 0.80 | 1.00 | `critical` / `quarantine` |
| one invisible char per keyword | **0.00** | **0.00** | **`none` / `none`** |

### Why the existing steps do not catch it

- The zero-width class covers only U+200B, U+200C, U+200D, U+2060 and U+FEFF.
- NFKD does not remove any of them.
- The combining-diacritic strip does not help: `unicodedata.combining()` returns `0` for U+FE0F and U+034F even though both are category `Mn`.
- No single category test works either. The usable characters span `Cf` (U+00AD), `Mn` (U+034F, U+FE0F), `Lo` (U+3164) and unassigned (U+2065).
- The existing `test_zero_width_characters` puts its zero-width space *between* words, where the whitespace collapse already handles it, so the mid-word case was never covered.

12 of 12 invisible characters I probed survived normalization: soft hyphen, combining grapheme joiner, arabic letter mark, mongolian vowel separator, left-to-right mark, right-to-left isolate, invisible plus, U+2065, hangul filler, variation selector 16, zero width no-break space, and a tag character.

### Impact

This is a complete detection bypass for the escalation, offensive-intent and composite paths. The evasion is one character per keyword and the characters are invisible in a normal editor or log viewer, so an operator reading the transcript sees the plain offensive text while the guardian reports nothing.

### Suggested fix

Define the set by the property that makes these characters usable for evasion instead of enumerating a handful: Unicode's `Default_Ignorable_Code_Point` ranges plus the invisible Hangul filler letters, stripped before the compatibility decomposition. `normalize_text` output is only ever matched against -- never returned to a caller or stored -- so stripping is safe here.

I have a fix and tests ready and will open a PR referencing this issue.

Contributor guide

Open the contributing guide

Research direction

Read agent-os/src/agent_os/integrations/conversation_guardian.py, starting with normalize_text and the -anchored detectors. Then inspect the existing test_zero_width_characters coverage and add regression coverage for invisible characters placed mid-word across the affected detection paths. Done means the reproduced evaded messages normalize and trigger the same alerts as their clean equivalents.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.