googleapis / googleapis/python-genai

Variance/jitter cannot be disabled for automatic retry/random exponential backoff

Open
#2,026 4 comments 0 reactions 1 assignee Claimed by @Venkaiahbabuneelam View on GitHub
priority: p2 type: bug type: feature request
Dominant language
Python
Stars
4k
Forks
1k
Avg merge
2d 11h
Merged PRs (30d)
40

Description

#### Bug description

When configuring automatic retry with random exponential backoff for `genai.Client` objects, disabling the variance/jitter of retry wait times is impossible using `genai.types.HttpRetryOptions` objects; setting the variance/jitter to `0.0` automatically defaults to `1.0` instead upon initialization of `genai.Client` objects.

The issue occurs in `google/genai/_api_client.py` in `retry_args()`:

`jitter=options.jitter or _RETRY_JITTER,`

The falsy check returns `False` when `0.0` is passed. Since `jitter` is typed and validated as `Float | None`, this line should instead be:

`jitter=options.jitter if options.jitter is not None else _RETRY_JITTER,`

or:

`jitter=_RETRY_JITTER if options.jitter is None else options.jitter,`

#### Expected behaviour

Initializing a `genai.Client` object with the `http_options.retry_options` parameter set to:

`genai.types.HttpRetryOptions(jitter = 0.0)`

should return a `genai.Client` object that has no variance/jitter added to retry wait times.

#### Actual behaviour

Initializing a `genai.Client` object with the `http_options.retry_options` parameter set to:

`genai.types.HttpRetryOptions(jitter = 0.0)`

returns a `genai.Client` object that is initialized to add a random value between 0 seconds and 1 second to retry wait times.

#### Steps to reproduce

1. Run the following code:

```
from google import genai

client = genai.Client(
vertexai=True,
project="foo",
location="foo",
http_options=genai.types.HttpOptions(
api_version="v1",
retry_options=genai.types.HttpRetryOptions(
jitter=0.0,
),
),
)

print(client._api_client._retry.wait.jitter)
```
The above block will print:
```
1
```

#### Environment details

- Programming language: Python
- OS: Any

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.