Issues with Gemini quickstart flow (retry behavior, model errors, and tool usage
- Dominant language
- Python
- Stars
- 73
- Forks
- 89
- Avg merge
- 14d 58m
- Merged PRs (30d)
- 2
Description
Hi, I found a few issues while testing the quickstart flow with Gemini that seem worth addressing for first-run usability.
Tagging @jackiekazil and @colinfrisch in case this is helpful context.
---
## **Observed issues**
### Unbounded retry behavior in ModuleLLM
`ModuleLLM.generate()` / `agenerate()` retry transient LiteLLM errors such as `APIConnectionError`, `Timeout`, and `RateLimitError` using exponential backoff, but without a clear stop condition.
In practice, this can make the script appear stuck while LiteLLM repeatedly prints its help/debug banner.
---
### Outdated Gemini model names causing 404 errors
Older Gemini model names such as:
- `gemini-pro`
- `gemini-1.5-pro`
now return 404 errors from the Google API.
This currently surfaces as a raw LiteLLM traceback, which makes it harder for users to understand that the issue is an outdated model name rather than a runtime failure.
---
### Missing actionable error message for NotFoundError
It may be helpful to rewrite Gemini `NotFoundError` into a more actionable message, for example:
> Model 'gemini/gemini-1.5-pro' was not found. This Gemini model may no longer be available via this API. Try 'gemini/gemini-2.0-flash' instead.
---
### Tool usage in quickstart (`tool_choice="required"`)
It may also be worth reviewing whether `tool_choice="required"` in the CoT executor step is ideal for minimal examples.
In practice, it can make simple quickstart usage more dependent on tool-calling behavior than users might expect, and may introduce unnecessary failures for providers that do not fully support tool calling.
---
## **Possible improvements**
- Add a bounded retry policy in `ModuleLLM` so transient failures eventually surface clearly (instead of potentially looping indefinitely).
- Rewrite Gemini `NotFoundError` responses into a more actionable error message with a suggested replacement model.
- Update quickstart/default examples to use a currently available Gemini model (e.g. `gemini-2.0-flash`, depending on API availability).
- Optionally revisit whether `tool_choice="required"` is appropriate for minimal quickstart examples.
---
## **Example of clearer error messaging**
Something like:
> Model 'gemini/gemini-1.5-pro' was not found. This Gemini model may no longer be available via this API. Try 'gemini/gemini-2.0-flash' instead.
---
If this direction sounds reasonable, I’d be happy to work on a fix and open a PR.
Contributor guide
Research direction
Start with ModuleLLM.generate() and agenerate() to trace the current retry behavior, then inspect the quickstart/default examples and the CoT executor's tool_choice="required" setting. Compare Gemini model handling for NotFoundError and define completion around bounded retries, actionable errors, current model examples, and an appropriate tool-use default.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100