NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

feature: GenerationLog option support in nemoguardrails with colang 2

Open
#1,108 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

duplicate enhancement
Dominant language
Python
Stars
7.2k
Forks
842
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Did you check the docs?
  • I have read all the NeMo-Guardrails docs
Is your feature request related to a problem? Please describe.

Currently, the GenerationOptions class includes a log option with fields like activated_rails, llm_calls, internal_events, and colang_history. These work in colang 1.x, but using them with Colang 2.0 raises a ValueError due to lack of support. This limits the or analyze the system when using newer Colang flows.

We rely on this logging capability to capture product-level metrics (e.g., which rails were activated, which certain actions were triggered, LLM calls, etc.) that are essential to our observability stack. Losing this in Colang 2.0 limits our ability to track user interactions, rail usage, and model behavior.

Describe the solution you'd like

We’d like support for the log option in GenerationOptions to be added for Colang 2.0.

Describe alternatives you've considered

Describe alternatives you've considered
Reverting to Colang 1.x.

Additional context

activated rails:

if (
options.log.activated_rails
or options.log.llm_calls
or options.log.internal_events
or options.log.colang_history
):
raise ValueError(
"The log option is not supported for Colang 2.0 configurations."
)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the GenerationOptions log handling and the Colang 2.0 validation that raises the shown ValueError. Trace how activated_rails, llm_calls, internal_events, and colang_history are handled in Colang 1.x, then determine what tests cover the equivalent Colang 2.0 behavior. Done means these logging options work for Colang 2.0 without the unsupported-option error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.