NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

bug: Chat streaming error handling bugs

Open
#1,771 3 comments 0 reactions 1 assignee View on GitHub

@ianliuy is already working on this.

Since Apr 14, 2026.

bug
Dominant language
Python
Stars
7.2k
Forks
842
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Did you check docs and existing issues?
  • I have read all the NeMo-Guardrails docs
  • I have updated the package to the latest version before submitting this issue
  • (optional) I have used the develop branch
  • I have searched the existing issues of NeMo-Guardrails
Python version (python --version)

Python 3.13.2

Operating system/version

Tahoe 26.3

NeMo-Guardrails version (if you must use a specific version and not the latest

0.21

Describe the bug

Two bugs discovered by Greptile in #1766 which were out-of-scope for that PR. Capturing here to be fixed later. Quoting from the comment:

Two P2 issues exist in the CLI streaming path worth addressing before or after merge.

Empty assistant message appended to history on streaming error (cli/chat.py lines 106-107): when a streaming error chunk is received and the loop breaks, bot_message_list is empty; the resulting {"role": "assistant", "content": ""} is unconditionally appended to history at line 181, corrupting subsequent turns with a blank assistant message.
Fragile error-chunk detection in CLI (cli/chat.py line 94): errors are identified via chunk.startswith('{"error"') — a raw string prefix check — while the server correctly uses ChunkError.model_validate_json(). Any whitespace or key-ordering change in the emitted JSON would silently pass the error through as normal content in the CLI.

Steps To Reproduce
  1. Run nemoguardrails chat application with streaming enabled.
  2. Insert streaming error chunk into App LLM response or with an "error" that doesn't match the expected format.
Expected Behavior

Handle error chunks gracefully and not corrupt conversation turns.

Actual Behavior

Corrupted conversation turns and potential missed error chunk responses

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.