googleapis / googleapis/python-genai
heavy calls die behind NAT idle-eviction because the default httpx transport doesn't set any TCP keepalive options
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
### Summary
`generate_content` can hang indefinitely (or raise a `ReadTimeout`) when a call runs longer than the TCP idle timeout of a NAT Gateway / firewall on the egress path. The default transport sets no TCP keepalive, so a connection that goes silent while awaiting the response gets dropped.
With no timeout set the read hangs forever; with one set it raises `ReadTimeout` at the deadline.
Likely the same root cause as #1893. For reference, the Anthropic Python SDK enables keepalive by default (`SO_KEEPALIVE` + `TCP_KEEPIDLE=60`), so it isn't affected: [`_base_client.py`](https://github.com/anthropics/anthropic-sdk-python/blob/main/src/anthropic/_base_client.py#L906-L930).
### Environment details
- Programming language: Python
- OS: Linux (container)
- Language runtime version: CPython 3.14
- Package version: google-genai 1.65.0
### Steps to reproduce
1. Run behind a NAT Gateway / stateful firewall with a short idle timeout (e.g. ~3-4 min).
2. Make a heavy `generate_content` call that stays silent for longer than that idle timeout.
3. With no read timeout: the call hangs forever. With one set: it raises a `ReadTimeout` at the deadline.
4. Enable TCP keepalive on a custom httpx transport (see below) & the issue disappears.
### Workaround
```python
import socket
import httpx
from google import genai
from google.genai import types
# Linux-specific socket options
_KEEPALIVE = [
(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1),
(socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, 60), # first probe after 60s idle
(socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, 15),
(socket.IPPROTO_TCP, socket.TCP_KEEPCNT, 4),
]
transport = httpx.HTTPTransport(socket_options=_KEEPALIVE)
client = genai.Client( # add vertexai=True, project=..., location=... for Vertex
http_options=types.HttpOptions(httpx_client=httpx.Client(transport=transport)),
)
```
Related: #1893
Contributor guide
Assessment
This issue has not been assessed yet.