microsoft / microsoft/agent-governance-toolkit

PII detection regexes scale quadratically on digit-heavy input

Open
#3,566 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
6.3k
Forks
1.1k
Avg merge
5d 11h
Merged PRs (30d)
142

Description

Observed while reviewing #3565 (present on main before that PR; timings identical at its merge-base).

`find_pii_matches()` in `agent-governance-python/agent-os/src/agent_os/credential_redactor.py` shows quadratic-ish scaling on pathological digit/IP-shaped input:

- `"1.1.1." * 8000` (~48KB) takes ~3.0s (phone/IP patterns)
- `"123-45-" * 8000` takes ~2.1s (SSN pattern)
- the `email_dots` secrets pattern takes ~5.4s on similar sizes

This path is reachable through the MCP gateway's PII detection on tool output, so a hostile tool can pad responses with digit soup to burn seconds of CPU per scan. Not a correctness issue (no bypass; the scan still completes), but worth bounding: input-length caps before the PII pass, atomic-group/possessive rewrites of the separator-heavy patterns, or a linear-time pre-filter (e.g. only run SSN/phone/CC patterns on windows around digit runs of plausible length).

A regression benchmark pinning worst-case scan time per KB would keep this from creeping.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in agent-governance-python/agent-os/src/agent_os/credential_redactor.py at find_pii_matches(), then reproduce the digit-heavy inputs and timings described here. Evaluate a bounded scan or regex/pre-filter change without reducing PII detection coverage. Add a regression benchmark that records worst-case scan time per KB and verify the scan no longer scales quadratically.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.