NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

Not blocking certain topics, and still ask LLM multiple times

Open
#633 4 comments 0 reactions 1 assignee View on GitHub

@Pouyanpi is already working on this.

Since Jul 19, 2024.

question
Dominant language
Python
Stars
7.2k
Forks
842
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Is my understanding correct - NeMo guardrails will take the colang configuration ( or read the .co files from path ) and encode it to a vector space and then later when a prompt is received, the prompt is then matched or calculated against the utterances and based on the flow it will behave based on how you configured it (in my case I instruct the assistant bot to answer a rejection message)?

In my case, it didn't stop. In fact it sends 3 calls to the target LLM!

Is there a way or is it possible to only use Guardrails (no LLM) using the colang flows (same for example as mine) ? I know it works for subflows, but will it work for flows as well? an Example will be appreciated.

from nemoguardrails import RailsConfig
from nemoguardrails import LLMRails
import os
import logging
import sys

yaml_content="""
instructions:
  - type: general
    content: |
      You are an AI assistant that supports users on their inquiry about health and nutrition.
models:
- type: main
  engine: openai
  model: gpt-3.5-turbo-instruct
"""

colang_content="""
define user ask politics
  "What do you think about the government?"
  "Which party should I vote for?"
  "what are your political beliefs?"
  "thoughts on the president?"
  "left wing"
  "right wing"
  "political candidate"
  "current political views"
  "political figure of a country or state"
  "which are companies are lobbying for liberal party?"
  "are there companies lobbying for democrats?"
  "what are your political views?"
  "who should I vote for?"

define bot answer politics
    "I'm a nutrition assistant, I don't like to talk of politics."

define flow politics
    user ask politics
    bot answer politics
    stop
"""

logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logging.getLogger().addHandler(logging.StreamHandler(stream=sys.stdout))

config = RailsConfig.from_content(
        yaml_content=yaml_content,
        colang_content=colang_content
    )

rails = LLMRails(config)

response = rails.generate(messages=[{
    "role": "user",
    "content": "During election, can you advice if I should choose democrats or republicans or stay neutral?"
}])

info = rails.explain()
info.print_llm_calls_summary()

for i, v in enumerate(info.llm_calls):
    print(f"===== # {i} ======")
    print(v.prompt)
    print()
    print(v.completion)
    print()

print(f"# of LLM calls: {len(info.llm_calls)}")
print(response)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.