MetPX / MetPX/wmo_mesh

telemetry/reporting support

Open
#12 3 comments 0 reactions 0 assignees View on GitHub
enhancement want_feedback
Dominant language
Python
Stars
4
Forks
2
PR merge metrics
No merged PRs in 30d

Description

Each data source typically likes to be aware of where their data has travelled (who has consumed it.) and if there were difficulties in processing it. Whenever an item is downloaded, one could provide a telemetry report back to the source to indicate how processing went.

In Sarracenia, this is implemented as the same format of messages, but travelling in the opposite direction (from the consumer back to the source, perhaps not by the same route.) Differences:

* instead of *v03.post*, the topic of a telemetry report is *v03.report*

* *content* header is removed.

* a report header is added, as an object:

* elapsedTime: the reported propagation delay based on the injection time received.
* resultCode: an httpd based numeric code indicating how it went.
* host: hostname submitting the report.
* user: broker username submitting the report.

* *from_cluster* is used to indicate where the report should end up. Each implementation will need to determine what node to send a report to in order to get it closer to *from_cluster*

In Sarracenia, reports are not yet deployed, as there is a fundamental problem that the number of reports will dwarf the number of posts (10:1 is a typical estimate), and so likely separate infrastructure needs to be established to avoid overloading the data delivery one. This emphasizes that the routing of telemetry will likely be different that the post messages.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the repository's Python MQTT pub/sub implementation and identify how Sarracenia-style post messages are represented and routed. The issue does not name files, tests, or an acceptance criterion; clarify the telemetry format, routing destination, and separate-infrastructure design before implementation can be considered done.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.