airqo-platform / airqo-platform/AirQo-api
No app-level timeout on synchronous BigQuery calls — large queries get killed by the worker
- Langage dominant
- JavaScript
- Étoiles
- 26
- Forks
- 24
- Merge moyen
- 5 h 36 min
- PR mergées (30 j)
- 81
Description
# No app-level timeout on synchronous BigQuery calls — large queries get killed by the worker
## Summary
Data-download queries run in-request via `.result().to_dataframe()` with no explicit timeout set, while gunicorn uses its default 30s worker timeout (no `--timeout` flag configured in the Dockerfile). A large/wide query that runs past 30s gets its worker killed mid-request, so the client sees a bare connection reset instead of a clean error response.
## Where this came from
Baalmart, following up on AirQo-frontend PR #3876's data-download review, tracing the request path in `src/analytics`. Flagged as a possible explanation for reported flakiness on repeated large requests.
## Affected code
- Synchronous BigQuery call — `bigquery_api.py:517-524`
- Gunicorn worker timeout (no explicit `--timeout` in the Dockerfile)
## Suggested fix
Set an explicit BigQuery job timeout with a clean error response when exceeded, and tune the gunicorn `--timeout` to give large exports room to complete (or fail cleanly) instead of being killed by the default.
## Acceptance criteria
- [ ] BigQuery data-download queries have an explicit timeout with a clean, client-visible error on expiry
- [ ] Gunicorn worker timeout is tuned to match, so large legitimate queries aren't killed mid-request
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Start by reading the synchronous BigQuery call at bigquery_api.py:517-524 and inspect the Dockerfile for the gunicorn command and its missing --timeout setting. Trace how expiry errors are returned to clients, then verify that data-download queries have a client-visible timeout and that the worker timeout allows legitimate large queries to complete or fail cleanly.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- python
- Domaine
- backend, cloud
- Type d'issue
- Bug
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- Calme
- Clarté
- Plutôt claire
- Accessibilité débutants
- 38/100