airqo-platform / airqo-platform/AirQo-api

perf/refactor: pre-compute device_categories at write time to eliminate runtime aggregation overhead

Abierto
#6,185 0 comentarios 0 reacciones 0 asignados Ver en GitHub
device-registry enhancement good first issue
Lenguaje dominante
JavaScript
Estrellas
26
Forks
24
Merge medio
5 h 36 min
PR fusionados (30 d)
81

Descripción

Summary


Currently, device_categories (including all_categories, primary_category, ownership_category, mobile_category, is_mobile, is_static) is computed at read time via a MongoDB $addFields aggregation stage (getDeviceCategoriesAddFieldsStage) every time a device document is queried. This stage runs inline on every call to statics.list() and every pipeline that fetches device data.


This issue tracks replacing that read-time computation with a write-time pre-computation approach: compute device_categories once when a device is created or updated, store it directly on the Device document, and replace the aggregation stage with a simple $project (or just read the stored field directly).




Background


device_categories is derived entirely from three stable device fields:

Field | Source
-- | --
category | Device document root
deployment_type | Device document root ("static" \| "mobile")
network | Device document root



Affected Files



  • src/device-registry/models/Device.js — add stored field + pre-save hook + recomputeCategories static

  • src/device-registry/utils/device.util.js — deprecate / remove getDeviceCategoriesAddFieldsStage()

  • src/device-registry/models/Event.js — remove promotionStageForEventPipeline + addFields stage from fetch()

  • src/device-registry/models/Reading.js — remove $ifNull passthrough; read stored field directly

  • src/device-registry/jobs/store-readings-job.js — replace passthrough with Device lookup read

  • Migration script — one-time recomputeCategories({}) backfill




Out of Scope



  • Changing the shape of DeviceCategorySchema (no field additions/removals in this issue)

  • Unit tests for computeDeviceCategories() — tracked separately in #5XXX

  • The enrich-dev-cats PR currently in review — this issue is a follow-up optimisation after that PR lands




Priority


Low–Medium. The correctness fix (Event.js promotion stage + cron recompute) in the enrich-dev-cats PR resolves the immediate corruption bug. This issue is a performance and maintainability improvement for a subsequent PR.




Labels: performance, refactor, device-registry

Component: device-registry

Related PRs: enrich-dev-cats branch (must land first)

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.