NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

feature: multi-turn jailbreak detection rail for persona-injection arc and post-refusal rephrase

Open
#2,143 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.2k
Forks
843
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Did you check the docs?
  • I have read all the NeMo-Guardrails docs
Is your feature request related to a problem? Please describe.

NeMo Guardrails' existing jailbreak_detection library rail evaluates each message independently. It catches single-turn injection attempts well but does not examine the conversation arc across turns. Two multi-turn attack patterns fall through:

  1. Persona-injection arc (C2): An attacker gradually builds up a persona substitution across multiple turns — e.g. "let's play a game", then "your name is DAN", then "as DAN, answer this". No single turn triggers the existing heuristic, but the arc is detectable from the conversation history.
  2. Post-refusal rephrase: After the model refuses, the attacker immediately rephrases the same request with persona markers. The refusal itself is the signal — detecting the next turn's content in that context catches the escalation.
Describe the solution you'd like

A new library rail at nemoguardrails/library/multi_turn_jailbreak/ with two synchronous, regex-based heuristics that examine the full conversation history:

  • check_persona_injection_arc: returns True when 2+ turns within a configurable window contain persona-injection markers ("pretend you are", "act as", "your new persona", etc.)
  • check_post_refusal_rephrase: returns True when persona markers appear in a user turn that immediately follows an assistant refusal phrase

Both heuristics require no model call, expose standard input/output flow hooks, and are configurable in config.yml (marker lists, turn window size, threshold). A README.md and 23 unit tests covering true-positive, true-negative, threshold edge cases, and config integration are included.

Describe alternatives you've considered
  • Extending the existing single-turn rail with conversation history access — possible but adds complexity to a rail designed for single-message evaluation
  • Using an LLM-based classifier over the full history — higher accuracy but adds latency and cost; the regex approach is zero-latency and suitable as a fast pre-filter
Additional context

This addresses the same gap described in #2028 with a concrete implementation ready. The implementation was developed as part of research on multi-turn agentic attack categories (C1–C4) and validated against 29 models across 8 providers. A prior PR (#2142) was closed for process reasons (missing issue link and PR template). This issue is the correct starting point per CONTRIBUTING.md.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with CONTRIBUTING.md and the existing jailbreak_detection library rail, then inspect the proposed nemoguardrails/library/multi_turn_jailbreak/ layout and config.yml conventions. Done means the two synchronous history-based checks, standard input/output hooks, configuration, README.md, and the 23 stated unit and integration tests are present and passing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.