NVIDIA-NeMo / NVIDIA-NeMo/Guardrails
feature: multi-message evaluation support in /v1/checks
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 842
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 25
Description
Did you check the docs?
- I have read all the NeMo-Guardrails docs
Is your feature request related to a problem? Please describe.
The /v1/checks endpoint (check_async) only evaluates the last user message in the messages array. When a client sends a chat completion request with conversation history containing multiple user messages, only the final one is checked against the configured rails.
This is a problem for gateway/proxy integrations (such as ai-gateway-payload-processing and Praxis) that need to validate all user messages in a request before forwarding it to the model.
The single-message limitation forces a workaround: splitting the request into N individual HTTP calls, one per user message.
I benchmarked this workaround against TrustyAI NeMo-Guardrails (develop HEAD, July 16) with PII detection rails and 1ms simulated pod-to-pod RTT:
User messages | Single call | N sequential calls | Overhead
1 | 39ms | 26ms | ~0%
5 | 142ms | 133ms | ~0%
10 | 223ms | 309ms | +39%
20 | 437ms | 597ms | +37% (+160ms)
At 10+ messages (typical for conversation history), the workaround adds ~37% latency purely from HTTP round-trip overhead.
Describe the solution you'd like
Add multi-message evaluation support to /v1/checks so that all user messages in the array are individually checked against the configured rails, not just the last one. The response should reflect the aggregate result:
passedif all messages passedblockedif any message was blocked (short-circuit)modifiedif any message was modified (with the transformed content)
This would let integrators use a single call to get both full message coverage and redaction support.
Describe alternatives you've considered
Split into N sequential HTTP calls - works but adds ~37% latency at 10+ messages and increases code complexity.
Additional context
Benchmark script
cc @m-misiura @christinaexyou @RobGeada @tgasser-nv @Pouyanpi
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Python implementation of the /v1/checks endpoint and its check_async entry point. Trace how the messages array is reduced to the last user message and inspect the existing response handling. Done means all user messages are evaluated, blocked results short-circuit, modified content is returned, and passed is reported only when all messages pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100