NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

bug: is_content_safe() drops policy categories for "safe" verdicts with trailing category tokens

Open
#2,174 1 comment 0 reactions 1 assignee View on GitHub

@Pouyanpi is already working on this.

Since Jul 15, 2026.

Dominant language
Python
Stars
7.2k
Forks
842
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Did you check docs and existing issues?
  • I have read all the NeMo-Guardrails docs
  • I have updated the package to the latest version before submitting this issue
  • I have used the develop branch
  • I have searched the existing issues of NeMo-Guardrails
Python version

N/A (logic bug, not environment-specific)

Operating system/version

N/A

NeMo-Guardrails version

develop (as of PR #2151)

Describe the bug

is_content_safe() in nemoguardrails/llm/output_parsers.py only extracts trailing policy-category tokens (e.g., S1, S8) when the response prefix is "unsafe". The "safe" branch unconditionally returns [True] and discards any trailing category text, so a response like "safe S1 S8" loses the category information entirely.

This was surfaced while asserting content_safety_check_input's RailOutcome.metadata["policy_violations"] in tests/test_content_safety_integration.py::test_content_safety_input_with_is_content_safe_parser_safe_with_violations, where the expected assertion result.metadata["policy_violations"] == ["S1", "S8"] had to be commented out because it can never pass with the current parser logic.

Steps To Reproduce
  1. Call is_content_safe("safe S1 S8").
  2. Observe the result is [True] instead of [True, "S1", "S8"].
  3. Run content_safety_check_input(...) with a mocked task manager returning this parsed result and inspect result.metadata["policy_violations"].
Expected Behavior

When a response is "safe" but still lists categories (an edge case, but currently unhandled), is_content_safe() should capture and return those categories rather than discarding them, so downstream consumers like content_safety_check_input/content_safety_check_output can populate policy_violations metadata correctly.

Actual Behavior

Trailing category tokens after a "safe" prefix are silently dropped; policy_violations metadata ends up empty ([]) even when categories were present in the raw LLM response.

Affected files/modules
  • nemoguardrails/llm/output_parsers.py (is_content_safe)
  • tests/test_content_safety_integration.py (commented-out assertion in test_content_safety_input_with_is_content_safe_parser_safe_with_violations)
References
Acceptance criteria
  • is_content_safe() parses trailing category tokens regardless of the "safe"/"unsafe" prefix, or documents/handles the "safe with violations" case explicitly.
  • The commented-out assertion in test_content_safety_input_with_is_content_safe_parser_safe_with_violations can be re-enabled and passes.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.