NVIDIA-NeMo / NVIDIA-NeMo/Guardrails
Issue with Integrating NeMo Guardrails with LangChain and LangServe for RAG Chatbot
@Pouyanpi is already working on this.
Since Nov 13, 2024.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 842
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 25
Description
Hello, I am currently working on a Retrieval-Augmented Generation (RAG) chatbot using LangChain and LangServe. I am attempting to integrate NeMo Guardrails into this system but am encountering issues with the chat history format when deploying my chain using LangServe.
Here is the detailed setup and issue:
### Setup
- LangChain Chain:
I created a LangChain chain which works perfectly in the Python environment.
I deployed it using LangServe, and the input JSON body for input looks like this:
- NeMo Guardrails Integration:
Integrated NeMo Guardrails with the conversational_retireval_chain using the following code:
chain_with_guardrails = RunnableRails(config, runnable=conversation_chain, input_key="question", output_key="answer")
return chain_with_guardrails
This worked well in the Python environment, with the following example running successfully:
chain_with_guardrails.invoke({
"question": "user question",
"chat_history": [("Hi", "Hello!"), ("Whats...", "Anwer...)]
})
### Issue
When deploying this with LangServe, I encountered the following issues:
{
"input": {
"question": "What is its latest release??",
"chat_history": [
["Hi, I am Ankur", "Hello"],
["What is AMP Impact?", "AMP impact?"]
]
},
"config": {},
"kwargs": {}
}
Error Recieved:
Error while execution generate_bot_message: Unsupported chat history format: <class 'list'>. Full chat history: [['Hi, I am Ankur', 'Hello'], ['What is AMP Impact?', 'AMP impact?']]
Attempt to Change Chat History to Tuples:
Changing each pair in the chat history to tuples, which worked in the Python environment, resulted in JSON parsing issues as JSON does not support tuples.
Input JSON Body:
{
"input": {
"question": "What is its latest release??",
"chat_history": [
("Hi, I am Ankur", "Hello"),
("What is AMP Impact?", "AMP impact?")
]
},
"config": {},
"kwargs": {}
}
Error Received:
{
"detail": [
"Invalid JSON body"
]
}
Empty Chat History:
Note: The chain works perfectly when the chat history is empty.
Request
I would like to request guidance on how to properly format the chat history for compatibility with NeMo Guardrails when deploying using LangServe. If there is a specific format or a workaround that I am missing, please let me know. Additionally, if this is an issue with NeMo Guardrails, I would appreciate any fixes or updates.
Thank you for your assistance.
Best regards,
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.