NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

Internal error occurs whenever I pass a prompt to conversation chain with NeMo

Open
#586 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
7.2k
Forks
843
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Can someone please assist me in solving the error:
I have a conversation chain set up with the help of langchain but the issue that keeps arising is that there is a internal error from the assistant's end and no other information to figure out what exactly went wrong. Some guidance would mean a lot.

Here's the code:
https://www.kaggle.com/code/beyondhorizon19/propex-9-0

Try prompting the model: tell me about colive 169 alpha

Here's the code snippet if you don't have kaggle:

bash
pipe = pipeline(
    "text-generation", 
    model=model, 
    tokenizer=tokenizer,
    framework='pt',
    max_new_tokens=1000,
    temperature=0.2,
)

llm = HuggingFacePipeline(pipeline=pipe)

HFPipelineMistral = get_llm_instance_wrapper(
    llm_instance=llm, llm_type="hf_pipeline_mistral_topic"
)
register_llm_provider("hf_pipeline_mistral_topic", HFPipelineMistral) 

colang_path = '/kaggle/input/hopeee/off_topic.co'
yaml_path = '/kaggle/input/hopeee/config.yml'
with open(colang_path, 'r') as file:
    colang_content = file.read()

with open(yaml_path, 'r') as file:
    yaml_content = file.read()

config = RailsConfig.from_content(yaml_content=yaml_content, colang_content=colang_content)
rails = LLMRails(config=config)

def get_best_match(query, choices):
    best_match, score = process.extractOne(query, choices)
    return best_match, score

def escape_curly_braces(context: str) -> str:
    return context.replace("{", "{{").replace("}", "}}")

async def converse(context,query):
  template = f"""Answer the question as truthfully as possible using the provided context. Do not make up answers.

  Stick to responding to the Human query only, do not generate extra question-answer pairs yourself! Your motto is to respond to whatever is being asked, that's all.

  If the user inputs random queries or remarks which have no relation with the context provided, respond with 'let us start over!'
  Reviews are in dictionary form with ratings and texts. Provide positive reviews when asked. 

  Be specific for questions seeking specific answers.
  Example:
  Human: What is the building gender type for Colive 178 Kingston?
  Bot Response: It is a female sharing only.

  For generic questions, give descriptive answers (not over 300 characters)

  Example:
  Human: Tell me about Colive 016 MB Paradise.
  Bot Response:
  Location: Colive 016 MB Paradise is in HAL 3rd Stage, New Thippasandra, Bangalore, near Cocoon Super Speciality Hospital. It's a prime area with pubs like Bootlegger 5 minutes away and Indira Nagar 100ft road 15 minutes away. Bagmane Tech Park is a 7-minute drive.
  Property Details: Modern 3 BHK apartment with vibrant colors and contemporary furnishings, offering single and double sharing options. Launched on September 26, 2017, managed by Colive, known for quick occupancy.
  Amenities: Includes 30 GB Wifi, bedside tables, cooking stoves, cupboards, dining tables, weekly housekeeping, geysers, kitchen, lift, no lock-in policy, power backup, premium bedding, refrigerators, security, SOS/Emergency Response, washing machines, exciting events, gas cylinders, DG & electricity at actuals.
  Pricing: Rents range from INR 8,500 to INR 17,000.
  Resident Feedback: Mostly positive. Appreciated for great services, excellent location, clean and vibrant rooms, and helpful staff.

  Here's the context: \n\n{context}

  Bot Response:
  """

  buffer_memory = ConversationBufferWindowMemory(k=3, return_messages=True) #maybe summary wala memory is better
  system_msg_template = SystemMessagePromptTemplate.from_template(template=template)
  human_msg_template = HumanMessagePromptTemplate.from_template(template="{input}")
  prompt_template = ChatPromptTemplate.from_messages([system_msg_template, MessagesPlaceholder(variable_name="history"), human_msg_template])
  
  chain = ConversationChain(memory=buffer_memory, prompt=prompt_template, llm=rails.llm, verbose=False)

  rails.register_action(chain, name="qa_chain")
  # return chain_with_guardrails

  history = [{"role": "user","content": query}]
  bot_message = await rails.generate_async(messages=history)
  return bot_message

property_names = df['Location Name'].to_list()
conversation_log = ""
current_context = ""

while True:
    user_query = input("User: ")
    
    if user_query.lower() == 'exit':
        print("Exiting...")
        break
    
    matched_property, score = get_best_match(user_query, property_names)
    
    if score > 60:
        start = time.time()
        
        matched_rows = df[df["Location Name"] == matched_property]
        current_context = matched_rows["combined_info"].values[0]
        current_context = escape_curly_braces(current_context)
        print("Property Identified...")
        
        bot_response=await converse(current_context,user_query)
        print(bot_response)
        
        end = time.time()
        execution_time=end-start
        print(f"Execution time: {execution_time} seconds")

    else:
        if current_context:
            start = time.time()
            conversation_log += f"User: {user_query}\nBot: {bot_response}\n\n"
            bot_response=await converse(current_context,user_query)
            print(bot_response)
            end = time.time()
        else:
            print("Property not found.")
            

bash

Output: {'role': 'assistant', 'content': "I'm sorry, an internal error has occurred."}

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked Kaggle notebook and the provided Python snippet, reproducing the prompt “tell me about colive 169 alpha” through LLMRails and the registered qa_chain. Inspect the RailsConfig inputs, custom pipeline, conversation chain, and returned assistant message to determine where the internal error originates. Done means identifying a reproducible cause and documenting a verified resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.