NVIDIA-NeMo / NVIDIA-NeMo/Guardrails
Internal error occurs whenever I pass a prompt to conversation chain with NeMo
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 843
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 25
Description
Can someone please assist me in solving the error:
I have a conversation chain set up with the help of langchain but the issue that keeps arising is that there is a internal error from the assistant's end and no other information to figure out what exactly went wrong. Some guidance would mean a lot.
Here's the code:
https://www.kaggle.com/code/beyondhorizon19/propex-9-0
Try prompting the model: tell me about colive 169 alpha
Here's the code snippet if you don't have kaggle:
bash
pipe = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
framework='pt',
max_new_tokens=1000,
temperature=0.2,
)
llm = HuggingFacePipeline(pipeline=pipe)
HFPipelineMistral = get_llm_instance_wrapper(
llm_instance=llm, llm_type="hf_pipeline_mistral_topic"
)
register_llm_provider("hf_pipeline_mistral_topic", HFPipelineMistral)
colang_path = '/kaggle/input/hopeee/off_topic.co'
yaml_path = '/kaggle/input/hopeee/config.yml'
with open(colang_path, 'r') as file:
colang_content = file.read()
with open(yaml_path, 'r') as file:
yaml_content = file.read()
config = RailsConfig.from_content(yaml_content=yaml_content, colang_content=colang_content)
rails = LLMRails(config=config)
def get_best_match(query, choices):
best_match, score = process.extractOne(query, choices)
return best_match, score
def escape_curly_braces(context: str) -> str:
return context.replace("{", "{{").replace("}", "}}")
async def converse(context,query):
template = f"""Answer the question as truthfully as possible using the provided context. Do not make up answers.
Stick to responding to the Human query only, do not generate extra question-answer pairs yourself! Your motto is to respond to whatever is being asked, that's all.
If the user inputs random queries or remarks which have no relation with the context provided, respond with 'let us start over!'
Reviews are in dictionary form with ratings and texts. Provide positive reviews when asked.
Be specific for questions seeking specific answers.
Example:
Human: What is the building gender type for Colive 178 Kingston?
Bot Response: It is a female sharing only.
For generic questions, give descriptive answers (not over 300 characters)
Example:
Human: Tell me about Colive 016 MB Paradise.
Bot Response:
Location: Colive 016 MB Paradise is in HAL 3rd Stage, New Thippasandra, Bangalore, near Cocoon Super Speciality Hospital. It's a prime area with pubs like Bootlegger 5 minutes away and Indira Nagar 100ft road 15 minutes away. Bagmane Tech Park is a 7-minute drive.
Property Details: Modern 3 BHK apartment with vibrant colors and contemporary furnishings, offering single and double sharing options. Launched on September 26, 2017, managed by Colive, known for quick occupancy.
Amenities: Includes 30 GB Wifi, bedside tables, cooking stoves, cupboards, dining tables, weekly housekeeping, geysers, kitchen, lift, no lock-in policy, power backup, premium bedding, refrigerators, security, SOS/Emergency Response, washing machines, exciting events, gas cylinders, DG & electricity at actuals.
Pricing: Rents range from INR 8,500 to INR 17,000.
Resident Feedback: Mostly positive. Appreciated for great services, excellent location, clean and vibrant rooms, and helpful staff.
Here's the context: \n\n{context}
Bot Response:
"""
buffer_memory = ConversationBufferWindowMemory(k=3, return_messages=True) #maybe summary wala memory is better
system_msg_template = SystemMessagePromptTemplate.from_template(template=template)
human_msg_template = HumanMessagePromptTemplate.from_template(template="{input}")
prompt_template = ChatPromptTemplate.from_messages([system_msg_template, MessagesPlaceholder(variable_name="history"), human_msg_template])
chain = ConversationChain(memory=buffer_memory, prompt=prompt_template, llm=rails.llm, verbose=False)
rails.register_action(chain, name="qa_chain")
# return chain_with_guardrails
history = [{"role": "user","content": query}]
bot_message = await rails.generate_async(messages=history)
return bot_message
property_names = df['Location Name'].to_list()
conversation_log = ""
current_context = ""
while True:
user_query = input("User: ")
if user_query.lower() == 'exit':
print("Exiting...")
break
matched_property, score = get_best_match(user_query, property_names)
if score > 60:
start = time.time()
matched_rows = df[df["Location Name"] == matched_property]
current_context = matched_rows["combined_info"].values[0]
current_context = escape_curly_braces(current_context)
print("Property Identified...")
bot_response=await converse(current_context,user_query)
print(bot_response)
end = time.time()
execution_time=end-start
print(f"Execution time: {execution_time} seconds")
else:
if current_context:
start = time.time()
conversation_log += f"User: {user_query}\nBot: {bot_response}\n\n"
bot_response=await converse(current_context,user_query)
print(bot_response)
end = time.time()
else:
print("Property not found.")
bash
Output: {'role': 'assistant', 'content': "I'm sorry, an internal error has occurred."}
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked Kaggle notebook and the provided Python snippet, reproducing the prompt “tell me about colive 169 alpha” through LLMRails and the registered qa_chain. Inspect the RailsConfig inputs, custom pipeline, conversation chain, and returned assistant message to determine where the internal error originates. Done means identifying a reproducible cause and documenting a verified resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100