NVIDIA-NeMo / NVIDIA-NeMo/Guardrails

Making Asynchronous API Requests - Colang 2.x in Firebase Environment

Open
#756 10 comments 0 reactions 1 assignee View on GitHub

@Pouyanpi is already working on this.

Since Sep 17, 2024.

enhancement
Dominant language
Python
Stars
7.2k
Forks
843
Avg merge
3d 1h
Merged PRs (30d)
25

Description

Context

I am encountering an issue while testing the NeMo Guardrails Framework Colang 2.x on a platform that utilizes a Firebase database. I have full control over events and thread IDs. My goal is to perform API requests to interact with my models within NeMo, similar to the capabilities in Colang 1.0, allowing asynchronous requests from different users with their respective userIds and threadIds.

Steps to Reproduce

1. System Configuration:

Operating System: Windows 10
NeMo Guardrails Version: 0.9.0
Colang Version: 2.x
Firebase: Used for managing events and threads
Virtual Machine: Compute Engine on Google Cloud Platform (GCP) for handling asynchronous requests

2. Functional Colang 1.0 Setup:

Utilizes asynchronous API and HTTP requests to communicate with actions.

flows.co:

define flow main

    user express start_automation_forms1
    bot express disparo_mensagem

    when user express positive
        bot ask how many leads
        when user respond positive
            bot ask losing sales because can't follow up
            when user respond positive
                execute desativa_bot(threadid=$threadid)
            else when user respond negative
                execute desativa_bot(threadid=$threadid)

actions.py:

@action()
async def desativa_bot(threadid):
    url = "https://example.net/desativaBot/" + threadid 

    async with httpx.AsyncClient() as client:
        try:
            response = await client.put(url)
            response.raise_for_status()  # Raises exception for HTTP error codes

            try:
                data = response.json()
            except ValueError:
                return {"error": "Invalid JSON response from API"}

            return data

        except httpx.RequestError as exc:
            return {"error": f"An error occurred while requesting {exc.request.url!r}."}
        except httpx.HTTPStatusError as exc:
            return {"error": f"Error response {exc.response.status_code} while requesting {exc.request.url!r}."}

Postman POST HTTP REQUEST:

{
    "config_id": "agent-c2",
    "document_id": "teste-v8.10",
    "thread_id": "teste-v8.10",
    "messages": [
        {
            "content": {
                "instancia": "testeMegav1",
                "threadid": "testeMegav1-phoneNumber"
            },
            "role": "context"
        },
        {
            "content": "olá",
            "role": "user"
        }
    ]
}

3. Testing with Colang 2.x:

Initial tests using nemoguardrails chat CLI worked flawlessly.
When executing the same requests via Postman, an error occurs after the first interaction.

Entered verbose mode.
<google.cloud.firestore_v1.client.Client object at 0x0000025E63DA8670>
Fetching 5 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<?, ?it/s]
Processing event StartFlow {'flow_id': 'main'}
Processing event {'type': 'ContextUpdate', 'data': {'instancia': 'testeMegav1', 'threadid': 'testeMegav1-phoneNumber'}}
Processing event {'type': 'UtteranceUserActionFinished', 'final_transcript': 'olá'}
UtteranceBotActionFinished
{"action_uid": "d6c4...", "action_name": "UtteranceBotAction", "status": "success", "is_success": true, "return_value": null, "events": [], "final_script": "Oi, tudo bem com voc\u00ea? Voc\u00ea usa automa\u00e7\u00e3o no WhatsApp?"}
Total processing took 0.15 seconds. LLM Stats: 0 total calls, 0 total time, 0 total tokens, 0 total prompt tokens, 0 total completion tokens, [] as latencies
INFO:     127.0.0.1:56319 - "POST /v1/chat/completions HTTP/1.1" 200 OK
ERROR:nemoguardrails.server.api:Providing `assistant` messages as input is not supported for Colang 2.0 configurations.
Traceback (most recent call last):
  File "C:\Users\Caio Moreno\Desktop\nemo-v3-production\venv\lib\site-packages\nemoguardrails\server\api.py", line 337, in chat_completion
    res = await llm_rails.generate_async(
  File "C:\Users\Caio Moreno\Desktop\nemo-v3-production\venv\lib\site-packages\nemoguardrails\rails\llm\llmrails.py", line 629, in generate_async
    events = self._get_events_for_messages(messages, state)
  File "C:\Users\Caio Moreno\Desktop\nemo-v3-production\venv\lib\site-packages\nemoguardrails\rails\llm\llmrails.py", line 490, in _get_events_for_messages       
    raise ValueError(
ValueError: Providing `assistant` messages as input is not supported for Colang 2.0 configurations.

Questions:

  1. Is this issue still considered a bug in Colang 2.x?
  2. Is there a way to perform asynchronous API requests in Colang 2.x similar to Colang 1.0, allowing multiple users with their respective userIds and threadIds?

Are there any additional configurations required to support asynchronous requests in the Colang 2.x environment?

Thank you very much for all support, and this incredible tool that is helping us to scale our solutions :)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.