googleapis / googleapis/python-genai
Enable (optional) concurrent execution of function calls in get_function_response_parts_async
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
### The problem:
- Relevant only for async client with enabled automatic function calling
- Currently, if a LLM returns two or more functions for execution, these will be executed sequentially.
- Considering these can often be blocking external API calls, the sequential wait for all of them to complete before returning can quickly mount, resulting in increased latency.
### Proposed solution
- Expand `AutomaticFunctionCallingConfig` and `AutomaticFunctionCallingConfigDict` with an optional parameter (e.g. `run_concurrently`), which defaults to `None` if not provided.
- Expand `get_function_response_parts_async` so it makes use of `asyncio.gather()` if `run_concurrently` is set to `True` .
### Client-side example
- In the end, it would look something like this on the client side:
```
response = await client.aio.models.generate_content(
model='gemini-2.5-flash',
contents='What is weather and traffic situation in Boston?',
config=types.GenerateContentConfig(
tools=[get_current_weather, get_current_traffic],
automatic_function_calling=types.AutomaticFunctionCallingConfig(run_concurrently=True)
),
)
```
Contributor guide
Assessment
This issue has not been assessed yet.