airqo-platform / airqo-platform/AirQo-api
perf/refactor: pre-compute device_categories at write time to eliminate runtime aggregation overhead
- Lenguaje dominante
- JavaScript
- Estrellas
- 26
- Forks
- 24
- Merge medio
- 5 h 36 min
- PR fusionados (30 d)
- 81
Descripción
Summary
Currently, device_categories (including all_categories, primary_category, ownership_category, mobile_category, is_mobile, is_static) is computed at read time via a MongoDB $addFields aggregation stage (getDeviceCategoriesAddFieldsStage) every time a device document is queried. This stage runs inline on every call to statics.list() and every pipeline that fetches device data.
This issue tracks replacing that read-time computation with a write-time pre-computation approach: compute device_categories once when a device is created or updated, store it directly on the Device document, and replace the aggregation stage with a simple $project (or just read the stored field directly).
Background
device_categories is derived entirely from three stable device fields:
Field | Source
-- | --
category | Device document root
deployment_type | Device document root ("static" \| "mobile")
network | Device document root
Affected Files
src/device-registry/models/Device.js— add stored field + pre-save hook +recomputeCategoriesstaticsrc/device-registry/utils/device.util.js— deprecate / removegetDeviceCategoriesAddFieldsStage()src/device-registry/models/Event.js— removepromotionStageForEventPipeline+ addFields stage fromfetch()src/device-registry/models/Reading.js— remove$ifNullpassthrough; read stored field directlysrc/device-registry/jobs/store-readings-job.js— replace passthrough with Device lookup read- Migration script — one-time
recomputeCategories({})backfill
Out of Scope
- Changing the shape of
DeviceCategorySchema(no field additions/removals in this issue) - Unit tests for
computeDeviceCategories()— tracked separately in #5XXX - The
enrich-dev-catsPR currently in review — this issue is a follow-up optimisation after that PR lands
Priority
Low–Medium. The correctness fix (Event.js promotion stage + cron recompute) in the enrich-dev-cats PR resolves the immediate corruption bug. This issue is a performance and maintainability improvement for a subsequent PR.
Labels: performance, refactor, device-registry
Component: device-registry
Related PRs: enrich-dev-cats branch (must land first)
Guía de contribución
Evaluación
Este issue todavía no se ha evaluado.