NVIDIA-NeMo / NVIDIA-NeMo/Guardrails
refactor: introduce a protocol-neutral internal error model
@Pouyanpi is already working on this.
Since Jul 28, 2026.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 842
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 25
Description
Problem
Provider clients and guardrails runtimes currently propagate failures through a
mix of exceptions, diagnostic strings, and OpenAI-shaped JSON. This couples
LLMRails, IORails, and provider integrations to one endpoint protocol and makes
it harder to preserve safe provider metadata consistently.
Internal code should not need to understand the public error envelope used by
/v1/chat/completions.
Proposed direction
Introduce a protocol-neutral typed error model or exception for failures
originating in provider clients, LLMRails, and IORails.
The representation should be able to retain:
- a safe client-facing message
- an internal error category
- an HTTP status, when one exists
- a provider code and parameter
- retry metadata
- provider response metadata needed for diagnostics
Diagnostic context that is unsafe for clients must remain separate from fields
that endpoint adapters may expose.
LLMRails, IORails, and provider clients should propagate this internal
representation rather than emitting OpenAI-shaped JSON as their control
representation. Public endpoints should translate it into their own protocol at
the server boundary.
Acceptance criteria
- A typed, protocol-neutral internal error representation is defined.
- Provider clients can populate safe messages, categories, statuses, provider
fields, retry metadata, and diagnostic metadata without constructing an
OpenAI error envelope. - LLMRails and IORails can propagate the representation without parsing
protocol-specific JSON. - OpenAI-compatible error rendering remains owned by the
/v1/chat/completionsserver boundary. - Unsafe diagnostic context cannot be serialized into a client response by
default. - Existing public API behavior remains compatible unless a separately approved
status-policy change requires otherwise. - Unit and integration tests cover construction, propagation, sanitization, and
endpoint rendering.
Compatibility notes
This is an internal representation change. Existing public APIs, including
stream_async(), should remain compatible. Migration may be incremental, but
temporary adapters must not create divergent LLMRails and IORails behavior.
Validation
- Characterize the current public HTTP and streaming behavior before migration.
- Test representative 400, 401, 404, 429, and 5xx provider failures.
- Test provider codes, parameters, retry metadata, and status-less transport
failures. - Test that diagnostic URLs, credentials, raw bodies, and internal identifiers
are not client-visible. - Run the related server, LLMRails, IORails, and provider-client test suites.
Parent epic
This issue is a sub-issue of:
epic: make downstream failures safe and reliable across Guardrails APIs
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.