Framework-wide telemetry service with OCPP device model and Async API integration

### Describe the problem

EVerest modules, hardware drivers in particular, produce data that is valuable for reporting but irrelevant to EVerest control flow: meter and rectifier temperatures, firmware states, vendor-specific error registers, communication statistics. This data is inherently **hardware-specific**: two powermeter drivers or power supplies have different data available and will therefire publish different properties. It therefore does not fit the EVerest interface/type system, which fixes variable sets at interface-definition time and requires explicit connection wiring between modules.

Four partial mechanisms exist today, and none is sufficient:

1. **Framework telemetry channel** (`telemetry_publish`): publish-only. There is no consumption API for other EVerest modules, nothing is enumerable (a consumer cannot learn what telemetry exists, let alone its types/units), and bindings are missing for Python/Rust.
2. **Interface variables**: typed and discoverable, but the variable set is frozen at interface-definition time, the opposite of hardware-specific data. Every consumer must be wired to every producer in every deployment config.
3. **Raw external MQTT**: individual drivers hand-roll their own telemetry publishers on ad-hoc topics; untyped, undiscoverable, reinvented per driver.
4. **Hand-wired OCPP device model values**: `EverestDeviceModelStorage` writes selected values (e.g. `EVSE`/`Power`) into the device model, but every new value is a hand-written C++ change, and these updates currently bypass variable monitoring entirely (no `NotifyEvent`).

The common root causes are: **no declaration** of telemetry (nothing enumerable, no types/units) and **no consumption path** for EVerest modules. As a result, hardware-specific data is not flexibly integrated and discoverable internally for EVerest modules as well as externally for e.g. AsyncAPI consumers.

### EVerest Domain

Framework, OCPP 1.6, OCPP 2.0.1, OCPP 2.1, Hardware Drivers, Other

### Affected Component

- `lib/everest/framework` (telemetry service, manifest schema, ConfigService) and `ev-cli` (codegen)
- `lib/everest/ocpp` (libocpp) and `lib/everest/ocpp_module_common` (device model integration)
- `modules/EVSE/OCPPmulti` (telemetry -> device model mapping)
- `modules/API/EVerestAPI` (new `telemetry_API` module)
- Hardware driver modules as producers (e.g. powermeter and power supply drivers)

### Describe your solution

A **framework-wide telemetry feature**, following the precedent of the two existing framework services that already bypass the requirements/connections system for many-producers/few-consumers, non-control-flow data: global error handling and the ConfigService.

### Requirements

1. **Easy publishing**: modules can declare and publish (partly hardware-specific) telemetry with minimal boilerplate, in any supported language; different modules publish different data.
2. **OCPP device model integration**: OCPP can consume telemetry and represent it as device model component/variables; readable (`GetVariables`/`GetReport`) **and monitorable** (`VariableMonitoring`/`NotifyEvent`), including delta and threshold monitors, for OCPP 2.x and (read access) OCPP 1.6.
3. **Async API exposure**: telemetry can be exposed on the `everest_api/...` channels for external consumers (cloud backends, dashboards, etc.), including schema discovery.
4. **Scalability**: the design must handle a high number of telemetry variables per station and frequent value updates without degrading the system.

Non-goals: telemetry never carries control-flow-relevant data (interfaces remain the mechanism for that), no delivery guarantees beyond MQTT QoS, no historical storage.

Three pillars:

1. **Declare** telemetry statically in the module's `manifest.yaml` (analogous to `config:`): a module declares one or more telemetry *sets*, each containing typed *entries* with metadata (type, unit, description, value list, min/max). Only the shape and keyword vocabulary are fixed; set and entry names are free identifiers chosen by the module authors. The declaration deliberately covers everything OCPP `VariableCharacteristics` needs. To avoid bloating `manifest.yaml` for modules with many entries, the manifest should be able to reference the whole telemetry declaration from a separate file next to the manifest.

```yaml
# manifest.yaml of a powermeter driver
telemetry: # fixed keyword: the new manifest section
livedata: # free identifier: a set id chosen by the module author
description: Live electrical measurements
entries: # fixed keyword: map of entry name -> entry schema
temperature_C: # free identifier: an entry name
type: number # fixed vocabulary: boolean | integer | number | string | object | array
unit: Celsius # optional -> OCPP VariableCharacteristics.unit
minimum: -40 # optional -> VariableCharacteristics.minLimit
maximum: 120 # optional -> VariableCharacteristics.maxLimit
fw_state:
type: string
values_list: [Idle, Measuring, Error] # optional -> VariableCharacteristics.valuesList
max_publish_rate_hz: 4 # optional: producer-side rate limit, enforced by the framework
diagnostics: # a module may declare several sets (e.g. fast livedata vs. slow diagnostics)
...
```

To keep `manifest.yaml` small for modules with many entries, the whole telemetry declaration can instead be referenced from a separate file next to the manifest:

```yaml
# manifest.yaml
telemetry:
$ref: telemetry.yaml
```

2. **Enumerate** all declared telemetry via the ConfigService (or a dedicated telemetry service): consumers can query the full telemetry catalog; including types and units; before any producer has published, which avoids startup races and makes the OCPP variable universe known at boot.

3. **Broadcast** values on a internal MQTT topic scheme with a framework-stamped envelope (module id/type, set, timestamp, module `mapping:` for EVSE/connector attribution) and a framework subscription API with filtering. Producers get a minimal publish call (optionally compile-time checked via a struct generated from the module's own manifest); consumers opt in via a manifest flag; adding a new hardware driver requires **zero** consumer-side configuration changes. Telemetry must not consume resources when nobody needs it and telemetry stays off per module unless enabled in the deployment configuration.

**Producer**: a typed struct generated from the module's own manifest, or the string-keyed baseline:

```cpp
// powermeter driver, anywhere in its poll loop
types::telemetry::Livedata data; // generated from THIS module's telemetry: section
data.temperature_C = 41.2; // typos / wrong types = compile errors
data.frequency_Hz = 49.98;
telemetry.publish(data); // only set members serialize (partial publish)

// string-keyed baseline (same wire): for ad-hoc entries and other language bindings, e.g. Python:
telemetry.publish_telemetry("livedata", {{"temperature_C", 41.2}});
```

**On the wire**: the module supplies only the set id and the `values`; the rest of the envelope is stamped by the framework:

```jsonc
// topic: everest-telemetry/v1/powermeter_1/livedata (QoS 0, not retained)
{
"version": 1, // envelope version (framework-stamped, independent of module data)
"module_id": "powermeter_1", // producing module instance
"module_type": "LemDCBM400600", // module type, for type-based filtering
"set": "livedata", // the declared telemetry set
"timestamp": "2026-08-12T10:41:07Z", // injected by the framework at publish time
"mapping": { "evse": 1 }, // the module's standard mapping: -> EVSE/connector attribution
"values": { // partial by design: carries what changed, not every declared entry
"temperature_C": 41.2,
"frequency_Hz": 49.98
}
}
```

**Consumer**: enumerate what exists (from the manifests, before any producer runs), then subscribe:

```cpp
// 1. Enumerate declared telemetry incl. types/units: served by the ConfigService
const auto definitions = config_service_client->get_telemetry_definitions();
const auto& entry = definitions.at("powermeter_1").sets.at("livedata").entries.at("temperature_C");
// entry.type == EntryType::number, entry.unit == "Celsius"

// 2. Subscribe to the value flow, filtered by module id / module type / set
TelemetryFilter filter;
filter.module_type = "LemDCBM400600";
subscribe_telemetry(filter, [](const TelemetryEnvelope& envelope) {
// envelope.module_id == "powermeter_1"
// envelope.set == "livedata"
// envelope.mapping.evse == 1
// envelope.values == { "temperature_C": 41.2, "frequency_Hz": 49.98 }
});
```

To keep producer-side effort minimal, shared hardware abstractions used by drivers (e.g. hardware APIs / driver base classes) should be extended to allow publishing telemetry through them, instead of each driver re-implementing the plumbing; derived/computed values remain the responsibility of the publishing module.

### OCPP device model integration

- **Curated mapping only**: only telemetry explicitly listed in a mapping file becomes device model variables; component/variable names come from the mapping, data type and unit from the declaration. The CSMS-facing surface changes only by explicit integrator decision, while the telemetry stream itself can evolve freely.
- **No SQLite writes per sample**: telemetry values live in a new in-memory device model storage source; only CSMS-set monitors are persisted across reboots. This addresses the performance concern with frequent telemetry updates.
- **Monitoring end-to-end**: `SetVariableMonitoring` (threshold, **delta**, periodic) works on telemetry variables and produces `NotifyEvent`. The changes required for this also fix the pre-existing gap where EVerest-sourced variable updates (e.g. `EVSE`/`Power`) never trigger monitors.
- OCPP 1.6 gets read access to mapped telemetry variables through the existing device-model-backed configuration key handling.

### Async API exposure

A new `telemetry_API` module republishes the telemetry stream on `everest_api/...` topics and serves the per-station definitions catalog for schema discovery. The external contract is a small, versioned envelope with a deliberately opaque `values` payload; external consumers introspect the catalog at runtime instead of compiling against per-field types. Lightweight monitoring rules (on-change, delta, thresholds, periodic) let external clients consume events instead of polling the raw stream.

### Alternatives considered

- **Typed generic telemetry interface** (new `interfaces/telemetry.yaml`, producers add a `provides:` implementation, consumers use 0..N requirements): rejected because every deployment would have to wire every producer to every consumer; exactly the friction the easy-publishing requirement forbids; and the typing benefit is largely maintain/extend for hardware-specific data (it degenerates to `object` blobs).
- **Raw MQTT bridge over the existing channels** (OCPP/API modules subscribe to the legacy telemetry topics, mapping file asserts types/units by hand): rejected because types/units would be hand-asserted per deployment and drift silently from producer code, and nothing is enumerable. Acceptable as a prototype; forward-compatible with the proposal if mappings address `{module_id, set, entry}`.

### Additional context

- Presented and discussed in the **Cloud Communication Working Group on 2026-08-12**; the working group agreed to move the proposal forward via this issue.
- Related:#2600 (OCPP <-> EVerest config mapping; the telemetry mapping extends this file family and should be coordinated with it), EVerest/EVerest#2496
- Rollout is staged and each stage is independently mergeable: framework (declaration, publish/subscribe, enumeration, bindings) -> OCPP integration -> Async API -> migration of reference producers (with a compatibility window for the legacy telemetry topics).

### Open Questions / Discussion

- type safety / auto-generation of telemetry payloads
- is flat string-keyword -> value representation complex enough?
- rate-control enforcements: in framework? module? consumer?
- OCPP integration: Only through explicit mapping or can be enabled automatically for EVerest module component-variable combination?
- Where are OCPP telemetry monitors stored? Existing EVerest device model database SQLite?

[Cloud_WG_2026-08-12.pdf](https://github.com/user-attachments/files/31223033/Cloud_WG_2026-08-12.pdf)

Start by reading the existing telemetry_publish and ConfigService implementations under lib/everest/framework, then inspect ev-cli, lib/everest/ocpp, lib/everest/ocpp_module_common, modules/EVSE/OCPPmulti, and modules/API/EVerestAPI. Done means the declared telemetry catalog, framework publish/subscribe path, curated OCPP device-model mapping, and Async API exposure work together while preserving monitoring behavior and scalability requirements.

Written by the indexing model from the issue text.