Cohort air-quality pollutant summary endpoint for AirQo Nexus dashboard cards

## Why

AirQo Nexus (Paul, frontend) needs a dashboard "summary card" widget that shows pollutant data for a **cohort of devices** — a logical group, not a single device — as one aggregated view: current value, min/max, trend vs. the previous comparable period, AQI category, and data coverage — for **pm2_5, pm10, and no2** — switchable client-side without one request per pollutant. This needs a single backend endpoint returning all three pollutants in one response.

**Aggregation semantics, stated explicitly:** every number in the response is computed across *all* devices currently in the cohort, combined — this is a group-level summary, not a per-device breakdown. Cohort membership is evaluated at request time and applied across the whole requested window; if a device joined the cohort partway through the window, its readings only exist from whenever it started reporting, so it naturally contributes a partial share of the window's data rather than being excluded or backfilled. This is not a per-device drill-down endpoint — if Nexus needs per-device values later, that's a separate, follow-up endpoint, not this one.

This is scoped entirely to `src/analytics` (BigQuery). Device-registry's 14-day readings TTL makes it unsuitable for dashboard-grade aggregation, and cohort membership/metadata there is unchanged by this work.

**This is new work, not a small addition** — nothing in the current codebase computes a "current/min/max/trend/AQI/coverage" summary, and there's no existing cohort → BigQuery pollutant-value path. Please read the whole issue before starting; the "Open design questions" section below has a few decisions worth confirming before writing code, to avoid rework.

## Current state (verified against `staging`, post rewrite-to-FastAPI)

- Analytics is now FastAPI (`main.py`), routes in `api/routers/v2.py` → service methods in `api/services/__init__.py` → BigQuery via `api/models/async_bigquery_api.py` (`AsyncBigQueryApi`, which delegates query-building 1:1 to the sync `api/models/bigquery_api.py::BigQueryApi` — do not fork query logic into the async file, per its own docstring).
- v2 base prefix: `/api/v2/analytics` (`config.py`, `main.py`).
- The closest existing pattern to imitate is `DashboardService._query_averages` / `get_device_exceedances` (`api/services/__init__.py:706-742` and `852-895`): a hand-built parameterized SQL string against `settings.bigquery_hourly_data`, filtered by `device_id IN UNNEST(@devices)`, executed via `AsyncBigQueryApi().execute_query_async(query, job_config)`.
- AQI/category helpers already exist and are unaffected by the rewrite: `get_pollutant_category`, `PM_COLOR_CATEGORY`, `PM_25_CATEGORY`/`PM_10_CATEGORY`/`NO2_CATEGORY`, `AQCSV_UNIT_MAPPER` — all in `api/utils/pollutants/pm_25.py`. Reuse these; don't redefine thresholds.
- **No cohort → device/site resolution exists for pollutant queries today.** The only cohort-aware BigQuery code is `api/models/summary_queries.py`, which joins BigQuery-mirrored `cohorts`/`cohorts_devices` tables (`config.py`'s `bigquery_cohorts`/`bigquery_cohorts_devices`) — but only to power the `/data/summary` **record-completeness** report, not pollutant values.
- Device-registry already exposes cohort → device/site resolution: **`GET /api/v2/devices/cohorts/:cohort_id/generate`** (`device-registry/utils/cohort.util.js:1025-1066`) — that's the full, externally-reachable path. ⚠️ **Two things to get right when calling it:**
- **Response shape**: the payload comes back under the key `sites_and_devices`, not `data`:
```json
{ "success": true, "message": "...", "sites_and_devices": { "device_ids": [...], "site_ids": [...] } }
```
(`device-registry/controllers/cohort.controller.js:40-59`, `handleResponse` renames `result.data` to whatever `key` was passed — here `"sites_and_devices"`.) `site_ids` is not deduplicated and can contain `null` for devices with no static site (mobile devices) — prefer `device_ids` for filtering to sidestep this.
- **`endpoint` argument if you use the existing `AirQoRequests` client** (`api/utils/http.py`): `AIRQO_API_BASE_URL` in `src/analytics/.env` is already `https://platform.airqo.net/api/v2` — it **already includes `/api/v2`**. The existing precedent in this codebase, `filter_non_private_sites_devices`, calls `AirQoRequests.request(endpoint="devices/cohorts/filterNonPrivateDevices", ...)` — no `/api/v2` prefix in the `endpoint` string, because the base URL supplies it. Follow the same pattern here: `endpoint="devices/cohorts/{cohort_id}/generate"` (relative, **no** `/api/v2`). Passing the full `/api/v2/devices/cohorts/...` path as `endpoint` will double up the prefix and 404.
- Once you have `device_ids`, the existing `device_ids` filter type is already supported by `BigQueryApi.build_filter_query`/`query_data` (`config.py`'s `FILTER_FIELD_MAPPING`) — no new filter type needed for the measurement query itself.
- `no2` is currently a second-class pollutant: excluded from `_VALID_POLLUTANTS`, `ExceedancesRequest`, and the `build_filter_query`/`query_data` pipeline used by chart/export endpoints. It's only accepted today in `DailyAveragesRequest`/`DeviceDailyAveragesRequest`'s raw-SQL path, where `request.pollutant` is interpolated directly as a column name. **Please confirm `no2` actually has populated data in the live `bigquery_hourly_data` table for lowcost devices before committing to it in the card design** — this is unverified from code alone.
- No response schema exists that's close to this shape — you'll be adding a new Pydantic model to `api/schemas/responses/__init__.py`.
- All timestamps/windows are UTC. `days` defines a rolling window relative to request time (`now - days` → `now`), not calendar-day-aligned — state this explicitly in the response/docs so the frontend doesn't assume midnight boundaries.

## Proposed endpoint contract (starting point — adjust as needed, see open questions)

```
GET /api/v2/analytics/dashboard/cohorts/{cohort_id}/summary?days=7
```

- `cohort_id` (path): the cohort's ID. Validate format before calling device-registry.
- `days` (query, optional, default 7, range 1–30): length of the current window; the comparison window is the immediately preceding period of the same length.
- No `tenant` param — this should operate on the single active platform tenant.

**Sample request:**

```
GET /api/v2/analytics/dashboard/cohorts/64f1b2c3d4e5f6a7b8c9d0e1/summary?days=7
```

No request body (GET).

**Response (200):**

```json
{
"status": "success",
"message": "Cohort pollutant summary successfully fetched",
"data": {
"cohort_id": "64f1b2c3d4e5f6a7b8c9d0e1",
"start_date": "2026-08-13T00:00:00Z",
"end_date": "2026-08-20T00:00:00Z",
"device_count": 12,
"reporting_device_count": 8,
"pollutants": {
"pm2_5": {
"value": 34.2,
"value_timestamp": "2026-08-20T00:00:00Z",
"unit": "µg/m³",
"min": 8.1,
"max": 112.4,
"percentage_change": -12.5,
"aqi_category": "Moderate",
"aqi_color": "#f8fe28",
"data_availability": {
"coverage_percentage": 87.3,
"status": "sufficient"
}
},
"pm10": { "...": "same shape" },
"no2": { "...": "same shape" }
}
}
}
```

- `device_count`: total devices currently in the cohort (the denominator for coverage).
- `reporting_device_count`: how many of those devices returned at least one reading in the current window — lets the frontend show e.g. "8 of 12 devices reporting" instead of only a coverage percentage.
- `value`: most recent hourly-average value across cohort devices within the window.
- `value_timestamp`: the timestamp of the hourly bucket `value` was computed from, so the frontend can show data freshness ("as of ...").
- `min`/`max`: across all devices/timestamps in the window.
- `percentage_change`: average(current window) vs. average(previous window); `null` if the previous window has no data.
- `aqi_category`/`aqi_color`: from `get_pollutant_category`/`PM_COLOR_CATEGORY`; `null` if `value` is `null`.
- `data_availability`: coverage percentage + a `sufficient`/`insufficient` flag so the frontend can show an empty/low-confidence state.
- A cohort with zero resolved devices should return `200` with `device_count: 0`, `reporting_device_count: 0`, and all pollutant fields `null`/`insufficient` — not an error. A nonexistent/invalid `cohort_id` should be a `400`.

## Reference: existing endpoints cited above (sample requests & responses)

**1. Device-registry cohort resolution — `GET /api/v2/devices/cohorts/{cohort_id}/generate`**, the endpoint you'll call to resolve `cohort_id` → `device_ids`.

Sample request:
```
GET /api/v2/devices/cohorts/64f1b2c3d4e5f6a7b8c9d0e1/generate
```
No request body. `tenant` query param is optional (defaults to `airqo`).

Sample response (200):
```json
{
"success": true,
"message": "Successfully returned the Site IDs and the Device IDs",
"sites_and_devices": {
"device_ids": ["64f1a2b3c4d5e6f7a8b9c0d1", "64f1a2b3c4d5e6f7a8b9c0d2"],
"site_ids": ["64f0a1b2c3d4e5f6a7b8c9d0", null]
}
}
```
Sample response, cohort not found (`device-registry/utils/cohort.util.js:1025-1066`) — `400`:
```json
{
"success": false,
"message": "Bad Request Errors",
"errors": { "message": "This Cohort does not exist" }
}
```

**2. Device-registry privacy check — `POST /api/v2/devices/cohorts/filterNonPrivateDevices`** — cited above only as the existing precedent for how this codebase already calls device-registry via `AirQoRequests` with a relative (no `/api/v2`) `endpoint` string; you likely won't call this one directly for the new feature.

Sample request:
```json
POST /api/v2/devices/cohorts/filterNonPrivateDevices
{
"device_ids": ["64f1a2b3c4d5e6f7a8b9c0d1", "64f1a2b3c4d5e6f7a8b9c0d2"]
}
```
Sample response (200):
```json
{
"success": true,
"message": "operation successful",
"devices": ["64f1a2b3c4d5e6f7a8b9c0d1"]
}
```

**3. Analytics — `POST /api/v2/analytics/data/summary` (cohort branch)** — the *only* existing cohort-aware analytics code today (`api/models/summary_queries.py`), cited above as the precedent for cohort → BigQuery joins. This is a record-completeness report, not a pollutant-value summary, and is not reused directly by the new endpoint — shown here so its shape is clear.

Sample request:
```json
POST /api/v2/analytics/data/summary
{
"startDateTime": "2026-08-13T00:00:00Z",
"endDateTime": "2026-08-20T00:00:00Z",
"cohort": "64f1b2c3d4e5f6a7b8c9d0e1"
}
```
Sample response (200):
```json
{
"status": "success",
"message": "successful",
"data": {
"cohort": "my-cohort-name",
"cohort_id": "64f1b2c3d4e5f6a7b8c9d0e1",
"hourly_records": 2016,
"calibrated_records": 1800,
"uncalibrated_records": 216,
"calibrated_percentage": 89.3,
"uncalibrated_percentage": 10.7,
"start_date_time": "2026-08-13T00:00:00Z",
"end_date_time": "2026-08-20T00:00:00Z",
"sites": [{ "site_id": "...", "site_name": "...", "hourly_records": 168, "...": "..." }],
"devices": [{ "device": "...", "hourly_records": 168, "...": "..." }]
},
"metadata": null
}
```
Sample response, no data found (200 — not an error):
```json
{
"status": "success",
"message": "No data found for cohort 64f1b2c3d4e5f6a7b8c9d0e1 from 2026-08-13T00:00:00Z to 2026-08-20T00:00:00Z",
"data": {},
"metadata": null
}
```

## Implementation notes

- **Trend needs two time windows.** No existing query builder does this in one call — either two sequential `execute_query_async` calls (current + previous), or one query with conditional aggregation (`CASE WHEN timestamp BETWEEN ... THEN ...`). Either is fine; pick whichever keeps the SQL readable.
- **Data coverage**: decide between reusing the precomputed `devices_summary` table (`api/models/device_summary_queries.py`, populated by a nightly batch job — cheap, but only cohort-joinable via the `summary_queries.py` pattern and may lag) vs. computing coverage inline from the same hourly-table query used for value/min/max/trend (simpler, always consistent with the other fields, heavier query). Our instinct is inline computation for consistency, but your call given the query-cost tradeoffs you can see that we can't.
- **Caching**: this endpoint will likely be polled repeatedly by multiple dashboard viewers looking at the same cohort. The service already has a Redis-backed cache (`api/utils/cache.py`) and a request-rate limiter (`api/middlewares/rate_limiter.py`, 100 req/min/IP) — consider a short server-side cache TTL (e.g. 1–5 min) on this endpoint's result rather than hitting BigQuery on every request. Not a hard requirement, but flag your decision either way in the PR.
- Follow `tests/conftest.py` conventions: `TestClient(app)`, mock BigQuery via `AsyncMock` on `AsyncBigQueryApi.execute_query_async` (see `tests/test_services.py`'s `test_get_summary_cohort_branch_includes_site_columns` around line 300 as a template for asserting both the generated SQL and the end-to-end response), and the `mock_privacy_filter` autouse fixture for any device-registry calls.
- Test cases to cover: cohort with full data, cohort with zero devices, cohort with partial coverage (e.g. no2 all-null for the window), invalid/nonexistent `cohort_id`, previous-window-has-no-data (percentage_change → null), and `reporting_device_count` < `device_count` (some cohort devices silent for the whole window).

## Open design questions — please confirm/decide before or while implementing

1. **Cohort resolution approach**: call device-registry's `/generate` endpoint live (fresher, adds a cross-service HTTP call + failure mode to handle) vs. querying the BigQuery-mirrored `cohorts`/`cohorts_devices` tables directly (no network call, but mirror freshness relative to Mongo is unknown — please check before relying on it). We'd lean toward the live device-registry call for correctness, but flag if the mirror tables are known to be fresh enough — that'd simplify things.
2. **Overall cohort-level AQI**: right now the frontend gets three independent `aqi_category` values (one per pollutant) and has to decide itself which one "drives" the card's overall status color. A common pattern (this is how the US EPA itself defines a location's AQI) is to also expose one overall category/color = the worst of the per-pollutant sub-categories. Please check with Paul whether Nexus wants this as a top-level `overall_aqi_category`/`overall_aqi_color` field — if yes, it's a small addition on top of the per-pollutant logic already in the contract above.
3. **AQI category labels**: `get_pollutant_category` returns AirQo's internal keys (`Good`, `Moderate`, `UHFSG`, `Unhealthy`, `VeryUnhealthy`, `Hazardous`). `UHFSG` (Unhealthy for Sensitive Groups) is a fairly inside-baseball label — please confirm with Paul whether Nexus expects these exact strings or a translated/expanded label.
4. **Privacy filtering**: existing dashboard/chart endpoints deliberately skip `_strip_private` (there's a "revisit before public cutover" TODO in `services/__init__.py`). Should this new endpoint follow that same convention, or apply filtering since it's a new endpoint? Worth a quick explicit decision rather than defaulting silently either way.
5. **`no2` data quality**: confirm real column population in `bigquery_hourly_data` for lowcost devices before finalizing — if it's sparse/unreliable, we may want to ship pm2_5/pm10 first and follow up on no2 separately.
6. **Health-standard comparison (WHO/national)** is explicitly **out of scope** for this issue — the endpoint only returns AQI category, not exceedance-vs-standard. Exceedance logic already exists (`ExceedancesRequest`/`STANDARDS_MAPPING`) and could power a follow-up card later; flagging so it isn't assumed to be bundled in here.

## Acceptance criteria

- [ ] New endpoint live at `GET /api/v2/analytics/dashboard/cohorts/{cohort_id}/summary`, no `tenant` param required.
- [ ] Response includes pm2_5, pm10, no2 in one payload, per the shape above (or an agreed adjustment), aggregated across the whole cohort (not per-device).
- [ ] Response includes both `device_count` and `reporting_device_count`, and `value_timestamp` per pollutant.
- [ ] Cohort with zero devices → `200`, not an error.
- [ ] Invalid/nonexistent `cohort_id` → `400` with a clear message.
- [ ] Tests added following `tests/` conventions, covering the cases listed above.
- [ ] Short API doc (path, params, response schema, example, error cases) — no tenant param — ready to hand to Paul.

Happy to hop on a call to walk through the investigation findings above if that's faster than back-and-forth in comments.

---