googleapis / googleapis/python-genai
Async file upload retry blocks the event loop
- Dominant language
- Python
- Stars
- 4k
- Forks
- 1k
- Avg merge
- 2d 11h
- Merged PRs (30d)
- 40
Description
#### Environment details
- Programming language: Python
- OS: macOS
- Language runtime version: 3.12.14
- Package version: 2.22.0 and `main` at `0ec3d8a4b2c85817434045dad739f6227c2d5c4c`
#### Steps to reproduce
1. Start a local HTTP server that returns `200` without `x-goog-upload-status` for the first resumable upload request, then returns `final` on the second request.
2. Call the HTTPX path used by `client.aio.files.upload` while a second coroutine sleeps for 50 ms.
3. Measure when the second coroutine resumes.
The local server avoids external API and quota dependencies. The upload retry is the real SDK and HTTPX path.
Reproducer
```python
import asyncio
import io
import threading
import time
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from google.genai import _api_client, types
class Handler(BaseHTTPRequestHandler):
attempts = 0
def do_POST(self):
type(self).attempts += 1
self.rfile.read(int(self.headers['Content-Length']))
self.send_response(200)
if type(self).attempts == 2:
self.send_header('X-Goog-Upload-Status', 'final')
self.end_headers()
def log_message(self, format, *args):
pass
async def main():
server = ThreadingHTTPServer(('127.0.0.1', 0), Handler)
threading.Thread(target=server.serve_forever, daemon=True).start()
base_url = f'http://127.0.0.1:{server.server_port}'
client = _api_client.BaseApiClient(
vertexai=False,
api_key='test',
http_options=types.HttpOptions(
base_url=base_url,
async_client_args={'trust_env': False},
),
)
client._use_aiohttp = lambda: False
_api_client.INITIAL_RETRY_DELAY = 0.25
started = time.monotonic()
async def ticker():
await asyncio.sleep(0.05)
return time.monotonic() - started
try:
_, tick = await asyncio.gather(
client._async_upload_fd(
io.BytesIO(b'test'), f'{base_url}/upload', 4
),
ticker(),
)
print(f'attempts={Handler.attempts}')
print(f'ticker_delay={tick:.3f}s')
print(f'event_loop_blocked={tick >= 0.20}')
finally:
await client.aclose()
server.shutdown()
server.server_close()
asyncio.run(main())
```
```text
attempts=2
ticker_delay=0.281s
event_loop_blocked=True
```
Cancellation is delayed by the same backoff:
```text
attempts=1
cancel_requested_after=0.433s
cancelled_after=0.434s
backoff_blocked_cancellation=True
```
The HTTPX branch in `_async_upload_fd` calls `time.sleep` during retry backoff. This blocks the event loop, so unrelated coroutines and cancellation cannot run until the sleep returns. The aiohttp branch already uses `await asyncio.sleep`.
Expected behavior: async upload retry backoff should yield to the event loop and remain cancellable.
Contributor guide
Assessment
This issue has not been assessed yet.