NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

epic: make downstream failures safe and reliable across Guardrails APIs

Open
#2,230 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

status: needs triage
Dominant language
Python
Stars
7.2k
Forks
842
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Goal

NeMo Guardrails should act as a trustworthy proxy: failures from generation
models, rail models, and rail APIs must be attributed correctly, represented
safely, and translated consistently for API clients.

Why

The current error paths do not always retain enough structure or provenance to
distinguish who caused a failure. For example:

  • A self-check rail model returning 429 can appear as if the API caller
    exhausted their own quota.
  • A model rejecting the server's API key can return 401 as if the caller
    supplied invalid credentials.
  • IORails can lose provider metadata or expose internal upstream details.
  • Serialized streaming error markers can be imitated by model output.
  • Operational rail failures do not produce complete observability records.

These behaviors make errors difficult for clients to interpret safely and
correctly.

Scope

Sub-issues under this epic should address:

  • failure provenance and outward status policy
  • generation-model versus rail-model and rail-API failures
  • server-owned versus caller-owned credentials and quotas
  • safe, structured errors across LLMRails and IORails
  • preservation of provider codes, parameters, and retry metadata
  • typed streaming errors that model-controlled content cannot forge
  • protocol-specific error rendering at the server boundary
  • complete observability for failed rail calls

Completion criteria

  • Client-attributable failures and server dependency failures are
    distinguishable.
  • Server-owned authentication or quota failures are not misrepresented as
    caller failures.
  • Client responses do not expose credentials, raw upstream bodies, internal
    URLs, or routing details.
  • Safe provider error metadata and retry information are preserved.
  • Model output cannot create a terminal streaming control event.
  • LLMRails and IORails follow the same approved error policy.
  • Public API endpoints render errors according to their own protocol contract.
  • Operational rail failures are represented correctly in spans and generation
    logs.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing error paths and public API boundaries for LLMRails and IORails, then separate the listed concerns into focused sub-issues. Done means applying one approved, consistent policy across generation, rail-model, and rail-API failures while meeting the stated safety, protocol-rendering, streaming, metadata, and observability criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend-api-design, observability, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.