airqo-platform / airqo-platform/AirQo-api

No app-level timeout on synchronous BigQuery calls — large queries get killed by the worker

Ouverte
#6,951 0 commentaires 0 réactions 1 personne assignée Assignée à @NicholasTurner23 Voir sur GitHub
analytics
Langage dominant
JavaScript
Étoiles
26
Forks
24
Merge moyen
5 h 36 min
PR mergées (30 j)
81

Description

# No app-level timeout on synchronous BigQuery calls — large queries get killed by the worker

## Summary
Data-download queries run in-request via `.result().to_dataframe()` with no explicit timeout set, while gunicorn uses its default 30s worker timeout (no `--timeout` flag configured in the Dockerfile). A large/wide query that runs past 30s gets its worker killed mid-request, so the client sees a bare connection reset instead of a clean error response.

## Where this came from
Baalmart, following up on AirQo-frontend PR #3876's data-download review, tracing the request path in `src/analytics`. Flagged as a possible explanation for reported flakiness on repeated large requests.

## Affected code
- Synchronous BigQuery call — `bigquery_api.py:517-524`
- Gunicorn worker timeout (no explicit `--timeout` in the Dockerfile)

## Suggested fix
Set an explicit BigQuery job timeout with a clean error response when exceeded, and tune the gunicorn `--timeout` to give large exports room to complete (or fail cleanly) instead of being killed by the default.

## Acceptance criteria
- [ ] BigQuery data-download queries have an explicit timeout with a clean, client-visible error on expiry
- [ ] Gunicorn worker timeout is tuned to match, so large legitimate queries aren't killed mid-request

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Start by reading the synchronous BigQuery call at bigquery_api.py:517-524 and inspect the Dockerfile for the gunicorn command and its missing --timeout setting. Trace how expiry errors are returned to clients, then verify that data-download queries have a client-visible timeout and that the worker timeout allows legitimate large queries to complete or fail cleanly.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python
Domaine
backend, cloud
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
Calme
Clarté
Plutôt claire
Accessibilité débutants
38/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.