NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

refactor: introduce a protocol-neutral internal error model

Open
#2,231 0 comments 0 reactions 1 assignee View on GitHub

@Pouyanpi is already working on this.

Since Jul 28, 2026.

refactoring status: needs triage
Dominant language
Python
Stars
7.2k
Forks
842
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Problem

Provider clients and guardrails runtimes currently propagate failures through a
mix of exceptions, diagnostic strings, and OpenAI-shaped JSON. This couples
LLMRails, IORails, and provider integrations to one endpoint protocol and makes
it harder to preserve safe provider metadata consistently.

Internal code should not need to understand the public error envelope used by
/v1/chat/completions.

Proposed direction

Introduce a protocol-neutral typed error model or exception for failures
originating in provider clients, LLMRails, and IORails.

The representation should be able to retain:

  • a safe client-facing message
  • an internal error category
  • an HTTP status, when one exists
  • a provider code and parameter
  • retry metadata
  • provider response metadata needed for diagnostics

Diagnostic context that is unsafe for clients must remain separate from fields
that endpoint adapters may expose.

LLMRails, IORails, and provider clients should propagate this internal
representation rather than emitting OpenAI-shaped JSON as their control
representation. Public endpoints should translate it into their own protocol at
the server boundary.

Acceptance criteria

  • A typed, protocol-neutral internal error representation is defined.
  • Provider clients can populate safe messages, categories, statuses, provider
    fields, retry metadata, and diagnostic metadata without constructing an
    OpenAI error envelope.
  • LLMRails and IORails can propagate the representation without parsing
    protocol-specific JSON.
  • OpenAI-compatible error rendering remains owned by the
    /v1/chat/completions server boundary.
  • Unsafe diagnostic context cannot be serialized into a client response by
    default.
  • Existing public API behavior remains compatible unless a separately approved
    status-policy change requires otherwise.
  • Unit and integration tests cover construction, propagation, sanitization, and
    endpoint rendering.

Compatibility notes

This is an internal representation change. Existing public APIs, including
stream_async(), should remain compatible. Migration may be incremental, but
temporary adapters must not create divergent LLMRails and IORails behavior.

Validation

  • Characterize the current public HTTP and streaming behavior before migration.
  • Test representative 400, 401, 404, 429, and 5xx provider failures.
  • Test provider codes, parameters, retry metadata, and status-less transport
    failures.
  • Test that diagnostic URLs, credentials, raw bodies, and internal identifiers
    are not client-visible.
  • Run the related server, LLMRails, IORails, and provider-client test suites.

Parent epic

This issue is a sub-issue of:

epic: make downstream failures safe and reliable across Guardrails APIs

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.