NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

Issue with Integrating NeMo Guardrails with LangChain and LangServe for RAG Chatbot

Open
#618 0 comments 0 reactions 1 assignee View on GitHub

@Pouyanpi is already working on this.

Since Nov 13, 2024.

bug
Dominant language
Python
Stars
7.2k
Forks
842
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Hello, I am currently working on a Retrieval-Augmented Generation (RAG) chatbot using LangChain and LangServe. I am attempting to integrate NeMo Guardrails into this system but am encountering issues with the chat history format when deploying my chain using LangServe.

Here is the detailed setup and issue:

### Setup

  1. LangChain Chain:

I created a LangChain chain which works perfectly in the Python environment.
I deployed it using LangServe, and the input JSON body for input looks like this:
image

  1. NeMo Guardrails Integration:

Integrated NeMo Guardrails with the conversational_retireval_chain using the following code:

chain_with_guardrails = RunnableRails(config, runnable=conversation_chain, input_key="question", output_key="answer") return chain_with_guardrails

This worked well in the Python environment, with the following example running successfully:
chain_with_guardrails.invoke({ "question": "user question", "chat_history": [("Hi", "Hello!"), ("Whats...", "Anwer...)] })

### Issue
When deploying this with LangServe, I encountered the following issues:

{ "input": { "question": "What is its latest release??", "chat_history": [ ["Hi, I am Ankur", "Hello"], ["What is AMP Impact?", "AMP impact?"] ] }, "config": {}, "kwargs": {} }

Error Recieved:

Error while execution generate_bot_message: Unsupported chat history format: <class 'list'>. Full chat history: [['Hi, I am Ankur', 'Hello'], ['What is AMP Impact?', 'AMP impact?']]

Attempt to Change Chat History to Tuples:

Changing each pair in the chat history to tuples, which worked in the Python environment, resulted in JSON parsing issues as JSON does not support tuples.

Input JSON Body:

{ "input": { "question": "What is its latest release??", "chat_history": [ ("Hi, I am Ankur", "Hello"), ("What is AMP Impact?", "AMP impact?") ] }, "config": {}, "kwargs": {} }

Error Received:

{ "detail": [ "Invalid JSON body" ] }

Empty Chat History:

Note: The chain works perfectly when the chat history is empty.

Request
I would like to request guidance on how to properly format the chat history for compatibility with NeMo Guardrails when deploying using LangServe. If there is a specific format or a workaround that I am missing, please let me know. Additionally, if this is an issue with NeMo Guardrails, I would appreciate any fixes or updates.

Thank you for your assistance.

Best regards,

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.