Cohort air-quality pollutant summary endpoint for AirQo Nexus dashboard cards

## Why

AirQo Nexus (Paul, frontend) needs a dashboard "summary card" widget that shows pollutant data for a **cohort of devices** — a logical group, not a single device — as one aggregated view: current value, min/max, trend vs. the previous comparable period, AQI category, and data coverage — for **pm2_5, pm10, and no2** — switchable client-side without one request per pollutant. This needs a single backend endpoint returning all three pollutants in one response.

**Aggregation semantics, stated explicitly:** every number in the response is computed across *all* devices currently in the cohort, combined — this is a group-level summary, not a per-device breakdown. Cohort membership is evaluated at request time and applied across the whole requested window; if a device joined the cohort partway through the window, its readings only exist from whenever it started reporting, so it naturally contributes a partial share of the window's data rather than being excluded or backfilled. This is not a per-device drill-down endpoint — if Nexus needs per-device values later, that's a separate, follow-up endpoint, not this one.

This is scoped entirely to `src/analytics` (BigQuery). Device-registry's 14-day readings TTL makes it unsuitable for dashboard-grade aggregation, and cohort membership/metadata there is unchanged by this work.

**This is new work, not a small addition** — nothing in the current codebase computes a "current/min/max/trend/AQI/coverage" summary, and there's no existing cohort → BigQuery pollutant-value path. Please read the whole issue before starting; the "Open design questions" section below has a few decisions worth confirming before writing code, to avoid rework.

## Current state (verified against `staging`, post rewrite-to-FastAPI)

- Analytics is now FastAPI (`main.py`), routes in `api/routers/v2.py` → service methods in `api/services/__init__.py` → BigQuery via `api/models/async_bigquery_api.py` (`AsyncBigQueryApi`, which delegates query-building 1:1 to the sync `api/models/bigquery_api.py::BigQueryApi` — do not fork query logic into the async file, per its own docstring).
- v2 base prefix: `/api/v2/analytics` (`config.py`, `main.py`).
- The closest existing pattern to imitate is `DashboardService._query_averages` / `get_device_exceedances` (`api/services/__init__.py:706-742` and `852-895`): a hand-built parameterized SQL string against `settings.bigquery_hourly_data`, filtered by `device_id IN UNNEST(@devices)`, executed via `AsyncBigQueryApi().execute_query_async(query, job_config)`.
- AQI/category helpers already exist and are unaffected by the rewrite: `get_pollutant_category`, `PM_COLOR_CATEGORY`, `PM_25_CATEGORY`/`PM_10_CATEGORY`/`NO2_CATEGORY`, `AQCSV_UNIT_MAPPER` — all in `api/utils/pollutants/pm_25.py`. Reuse these; don't redefine thresholds.
- **No cohort → device/site resolution exists for pollutant queries today.** The only cohort-aware BigQuery code is `api/models/summary_queries.py`, which joins BigQuery-mirrored `cohorts`/`cohorts_devices` tables (`config.py`'s `bigquery_cohorts`/`bigquery_cohorts_devices`) — but only to power the `/data/summary` **record-completeness** report, not pollutant values.
- Device-registry already exposes cohort → device/site resolution: **`GET /api/v2/devices/cohorts/:cohort_id/generate`** (`device-registry/utils/cohort.util.js:1025-1066`) — that's the full, externally-reachable path. ⚠️ **Two things to get right when calling it:**
- **Response shape**: the payload comes back under the key `sites_and_devices`, not `data`:
```json
{ "success": true, "message": "...", "sites_and_devices": { "device_ids": [...], "site_ids": [...] } }
```
(`device-registry/controllers/cohort.controller.js:40-59`, `handleResponse` renames `result.data` to whatever `key` was passed — here `"sites_and_devices"`.) `site_ids` is not deduplicated and can contain `null` for devices with no static site (mobile devices) — prefer `device_ids` for filtering to sidestep this.
- **`endpoint` argument if you use the existing `AirQoRequests` client** (`api/utils/http.py`): `AIRQO_API_BASE_URL` in `src/analytics/.env` is already `https://platform.airqo.net/api/v2` — it **already includes `/api/v2`**. The existing precedent in this codebase, `filter_non_private_sites_devices`, calls `AirQoRequests.request(endpoint="devices/cohorts/filterNonPrivateDevices", ...)` — no `/api/v2` prefix in the `endpoint` string, because the base URL supplies it. Follow the same pattern here: `endpoint="devices/cohorts/{cohort_id}/generate"` (relative, **no** `/api/v2`). Passing the full `/api/v2/devices/cohorts/...` path as `endpoint` will double up the prefix and 404.
- Once you have `device_ids`, the existing `device_ids` filter type is already supported by `BigQueryApi.build_filter_query`/`query_data` (`config.py`'s `FILTER_FIELD_MAPPING`) — no new filter type needed for the measurement query itself.
- `no2` is currently a second-class pollutant: excluded from `_VALID_POLLUTANTS`, `ExceedancesRequest`, and the `build_filter_query`/`query_data` pipeline used by chart/export endpoints. It's only accepted today in `DailyAveragesRequest`/`DeviceDailyAveragesRequest`'s raw-SQL path, where `request.pollutant` is interpolated directly as a column name. **Please confirm `no2` actually has populated data in the live `bigquery_hourly_data` table for lowcost devices before committing to it in the card design** — this is unverified from code alone.
- No response schema exists that's close to this shape — you'll be adding a new Pydantic model to `api/schemas/responses/__init__.py`.
- All timestamps/windows are UTC. `days` defines a rolling window relative to request time (`now - days` → `now`), not calendar-day-aligned — state this explicitly in the response/docs so the frontend doesn't assume midnight boundaries.

## Proposed endpoint contract (starting point — adjust as needed, see open questions)

```
GET /api/v2/analytics/dashboard/cohorts/{cohort_id}/summary?days=7
```

- `cohort_id` (path): the cohort's ID. Validate format before calling device-registry.
- `days` (query, optional, default 7, range 1–30): length of the current window; the comparison window is the immediately preceding period of the same length.
- No `tenant` param — this should operate on the single active platform tenant.

**Sample request:**

```
GET /api/v2/analytics/dashboard/cohorts/64f1b2c3d4e5f6a7b8c9d0e1/summary?days=7
```

No request body (GET).

**Response (200):**

```json
{
"status": "success",
"message": "Cohort pollutant summary successfully fetched",
"data": {
"cohort_id": "64f1b2c3d4e5f6a7b8c9d0e1",
"start_date": "2026-08-13T00:00:00Z",
"end_date": "2026-08-20T00:00:00Z",
"device_count": 12,
"reporting_device_count": 8,
"pollutants": {
"pm2_5": {
"value": 34.2,
"value_timestamp": "2026-08-20T00:00:00Z",
"unit": "µg/m³",
"min": 8.1,
"max": 112.4,
"percentage_change": -12.5,
"aqi_category": "Moderate",
"aqi_color": "#f8fe28",
"data_availability": {
"coverage_percentage": 87.3,
"status": "sufficient"
}
},
"pm10": { "...": "same shape" },
"no2": { "...": "same shape" }
}
}
}
```

- `device_count`: total devices currently in the cohort (the denominator for coverage).
- `reporting_device_count`: how many of those devices returned at least one reading in the current window — lets the frontend show e.g. "8 of 12 devices reporting" instead of only a coverage percentage.
- `value`: most recent hourly-average value across cohort devices within the window.
- `value_timestamp`: the timestamp of the hourly bucket `value` was computed from, so the frontend can show data freshness ("as of ...").
- `min`/`max`: across all devices/timestamps in the window.
- `percentage_change`: average(current window) vs. average(previous window); `null` if the previous window has no data.
- `aqi_category`/`aqi_color`: from `get_pollutant_category`/`PM_COLOR_CATEGORY`; `null` if `value` is `null`.
- `data_availability`: coverage percentage + a `sufficient`/`insufficient` flag so the frontend can show an empty/low-confidence state.
- A cohort with zero resolved devices should return `200` with `device_count: 0`, `reporting_device_count: 0`, and all pollutant fields `null`/`insufficient` — not an error. A nonexistent/invalid `cohort_id` should be a `400`.

## Reference: existing endpoints cited above (sample requests & responses)

**1. Device-registry cohort resolution — `GET /api/v2/devices/cohorts/{cohort_id}/generate`**, the endpoint you'll call to resolve `cohort_id` → `device_ids`.

Sample request:
```
GET /api/v2/devices/cohorts/64f1b2c3d4e5f6a7b8c9d0e1/generate
```
No request body. `tenant` query param is optional (defaults to `airqo`).

Sample response (200):
```json
{
"success": true,
"message": "Successfully returned the Site IDs and the Device IDs",
"sites_and_devices": {
"device_ids": ["64f1a2b3c4d5e6f7a8b9c0d1", "64f1a2b3c4d5e6f7a8b9c0d2"],
"site_ids": ["64f0a1b2c3d4e5f6a7b8c9d0", null]
}
}
```
Sample response, cohort not found (`device-registry/utils/cohort.util.js:1025-1066`) — `400`:
```json
{
"success": false,
"message": "Bad Request Errors",
"errors": { "message": "This Cohort does not exist" }
}
```

**2. Device-registry privacy check — `POST /api/v2/devices/cohorts/filterNonPrivateDevices`** — cited above only as the existing precedent for how this codebase already calls device-registry via `AirQoRequests` with a relative (no `/api/v2`) `endpoint` string; you likely won't call this one directly for the new feature.

Sample request:
```json
POST /api/v2/devices/cohorts/filterNonPrivateDevices
{
"device_ids": ["64f1a2b3c4d5e6f7a8b9c0d1", "64f1a2b3c4d5e6f7a8b9c0d2"]
}
```
Sample response (200):
```json
{
"success": true,
"message": "operation successful",
"devices": ["64f1a2b3c4d5e6f7a8b9c0d1"]
}
```

**3. Analytics — `POST /api/v2/analytics/data/summary` (cohort branch)** — the *only* existing cohort-aware analytics code today (`api/models/summary_queries.py`), cited above as the precedent for cohort → BigQuery joins. This is a record-completeness report, not a pollutant-value summary, and is not reused directly by the new endpoint — shown here so its shape is clear.

Sample request:
```json
POST /api/v2/analytics/data/summary
{
"startDateTime": "2026-08-13T00:00:00Z",
"endDateTime": "2026-08-20T00:00:00Z",
"cohort": "64f1b2c3d4e5f6a7b8c9d0e1"
}
```
Sample response (200):
```json
{
"status": "success",
"message": "successful",
"data": {
"cohort": "my-cohort-name",
"cohort_id": "64f1b2c3d4e5f6a7b8c9d0e1",
"hourly_records": 2016,
"calibrated_records": 1800,
"uncalibrated_records": 216,
"calibrated_percentage": 89.3,
"uncalibrated_percentage": 10.7,
"start_date_time": "2026-08-13T00:00:00Z",
"end_date_time": "2026-08-20T00:00:00Z",
"sites": [{ "site_id": "...", "site_name": "...", "hourly_records": 168, "...": "..." }],
"devices": [{ "device": "...", "hourly_records": 168, "...": "..." }]
},
"metadata": null
}
```
Sample response, no data found (200 — not an error):
```json
{
"status": "success",
"message": "No data found for cohort 64f1b2c3d4e5f6a7b8c9d0e1 from 2026-08-13T00:00:00Z to 2026-08-20T00:00:00Z",
"data": {},
"metadata": null
}
```

## Implementation notes

- **Trend needs two time windows.** No existing query builder does this in one call — either two sequential `execute_query_async` calls (current + previous), or one query with conditional aggregation (`CASE WHEN timestamp BETWEEN ... THEN ...`). Either is fine; pick whichever keeps the SQL readable.
- **Data coverage**: decide between reusing the precomputed `devices_summary` table (`api/models/device_summary_queries.py`, populated by a nightly batch job — cheap, but only cohort-joinable via the `summary_queries.py` pattern and may lag) vs. computing coverage inline from the same hourly-table query used for value/min/max/trend (simpler, always consistent with the other fields, heavier query). Our instinct is inline computation for consistency, but your call given the query-cost tradeoffs you can see that we can't.
- **Caching**: this endpoint will likely be polled repeatedly by multiple dashboard viewers looking at the same cohort. The service already has a Redis-backed cache (`api/utils/cache.py`) and a request-rate limiter (`api/middlewares/rate_limiter.py`, 100 req/min/IP) — consider a short server-side cache TTL (e.g. 1–5 min) on this endpoint's result rather than hitting BigQuery on every request. Not a hard requirement, but flag your decision either way in the PR.
- Follow `tests/conftest.py` conventions: `TestClient(app)`, mock BigQuery via `AsyncMock` on `AsyncBigQueryApi.execute_query_async` (see `tests/test_services.py`'s `test_get_summary_cohort_branch_includes_site_columns` around line 300 as a template for asserting both the generated SQL and the end-to-end response), and the `mock_privacy_filter` autouse fixture for any device-registry calls.
- Test cases to cover: cohort with full data, cohort with zero devices, cohort with partial coverage (e.g. no2 all-null for the window), invalid/nonexistent `cohort_id`, previous-window-has-no-data (percentage_change → null), and `reporting_device_count` < `device_count` (some cohort devices silent for the whole window).

## Open design questions — please confirm/decide before or while implementing

1. **Cohort resolution approach**: call device-registry's `/generate` endpoint live (fresher, adds a cross-service HTTP call + failure mode to handle) vs. querying the BigQuery-mirrored `cohorts`/`cohorts_devices` tables directly (no network call, but mirror freshness relative to Mongo is unknown — please check before relying on it). We'd lean toward the live device-registry call for correctness, but flag if the mirror tables are known to be fresh enough — that'd simplify things.
2. **Overall cohort-level AQI**: right now the frontend gets three independent `aqi_category` values (one per pollutant) and has to decide itself which one "drives" the card's overall status color. A common pattern (this is how the US EPA itself defines a location's AQI) is to also expose one overall category/color = the worst of the per-pollutant sub-categories. Please check with Paul whether Nexus wants this as a top-level `overall_aqi_category`/`overall_aqi_color` field — if yes, it's a small addition on top of the per-pollutant logic already in the contract above.
3. **AQI category labels**: `get_pollutant_category` returns AirQo's internal keys (`Good`, `Moderate`, `UHFSG`, `Unhealthy`, `VeryUnhealthy`, `Hazardous`). `UHFSG` (Unhealthy for Sensitive Groups) is a fairly inside-baseball label — please confirm with Paul whether Nexus expects these exact strings or a translated/expanded label.
4. **Privacy filtering**: existing dashboard/chart endpoints deliberately skip `_strip_private` (there's a "revisit before public cutover" TODO in `services/__init__.py`). Should this new endpoint follow that same convention, or apply filtering since it's a new endpoint? Worth a quick explicit decision rather than defaulting silently either way.
5. **`no2` data quality**: confirm real column population in `bigquery_hourly_data` for lowcost devices before finalizing — if it's sparse/unreliable, we may want to ship pm2_5/pm10 first and follow up on no2 separately.
6. **Health-standard comparison (WHO/national)** is explicitly **out of scope** for this issue — the endpoint only returns AQI category, not exceedance-vs-standard. Exceedance logic already exists (`ExceedancesRequest`/`STANDARDS_MAPPING`) and could power a follow-up card later; flagging so it isn't assumed to be bundled in here.

## Acceptance criteria

- [ ] New endpoint live at `GET /api/v2/analytics/dashboard/cohorts/{cohort_id}/summary`, no `tenant` param required.
- [ ] Response includes pm2_5, pm10, no2 in one payload, per the shape above (or an agreed adjustment), aggregated across the whole cohort (not per-device).
- [ ] Response includes both `device_count` and `reporting_device_count`, and `value_timestamp` per pollutant.
- [ ] Cohort with zero devices → `200`, not an error.
- [ ] Invalid/nonexistent `cohort_id` → `400` with a clear message.
- [ ] Tests added following `tests/` conventions, covering the cases listed above.
- [ ] Short API doc (path, params, response schema, example, error cases) — no tenant param — ready to hand to Paul.

Happy to hop on a call to walk through the investigation findings above if that's faster than back-and-forth in comments.

---

Start in src/analytics/main.py, api/routers/v2.py, api/services/__init__.py, and the BigQuery model files; review the existing DashboardService queries and the device-registry cohort generate response. Confirm no2 data availability and the open design decisions before implementation. Done means the new endpoint follows the proposed response and window semantics, including empty cohorts and insufficient coverage.

由索引模型根据 Issue 内容生成。