microsoft / microsoft/agent-governance-toolkit
PII detection regexes scale quadratically on digit-heavy input
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.3k
- Forks
- 1.1k
- Avg merge
- 5d 11h
- Merged PRs (30d)
- 142
Description
Observed while reviewing #3565 (present on main before that PR; timings identical at its merge-base).
`find_pii_matches()` in `agent-governance-python/agent-os/src/agent_os/credential_redactor.py` shows quadratic-ish scaling on pathological digit/IP-shaped input:
- `"1.1.1." * 8000` (~48KB) takes ~3.0s (phone/IP patterns)
- `"123-45-" * 8000` takes ~2.1s (SSN pattern)
- the `email_dots` secrets pattern takes ~5.4s on similar sizes
This path is reachable through the MCP gateway's PII detection on tool output, so a hostile tool can pad responses with digit soup to burn seconds of CPU per scan. Not a correctness issue (no bypass; the scan still completes), but worth bounding: input-length caps before the PII pass, atomic-group/possessive rewrites of the separator-heavy patterns, or a linear-time pre-filter (e.g. only run SSN/phone/CC patterns on windows around digit runs of plausible length).
A regression benchmark pinning worst-case scan time per KB would keep this from creeping.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in agent-governance-python/agent-os/src/agent_os/credential_redactor.py at find_pii_matches(), then reproduce the digit-heavy inputs and timings described here. Evaluate a bounded scan or regex/pre-filter change without reducing PII detection coverage. Add a regression benchmark that records worst-case scan time per KB and verify the scan no longer scales quadratically.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100