telemetry/reporting support
- Dominant language
- Python
- Stars
- 4
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
Each data source typically likes to be aware of where their data has travelled (who has consumed it.) and if there were difficulties in processing it. Whenever an item is downloaded, one could provide a telemetry report back to the source to indicate how processing went.
In Sarracenia, this is implemented as the same format of messages, but travelling in the opposite direction (from the consumer back to the source, perhaps not by the same route.) Differences:
* instead of *v03.post*, the topic of a telemetry report is *v03.report*
* *content* header is removed.
* a report header is added, as an object:
* elapsedTime: the reported propagation delay based on the injection time received.
* resultCode: an httpd based numeric code indicating how it went.
* host: hostname submitting the report.
* user: broker username submitting the report.
* *from_cluster* is used to indicate where the report should end up. Each implementation will need to determine what node to send a report to in order to get it closer to *from_cluster*
In Sarracenia, reports are not yet deployed, as there is a fundamental problem that the number of reports will dwarf the number of posts (10:1 is a typical estimate), and so likely separate infrastructure needs to be established to avoid overloading the data delivery one. This emphasizes that the routing of telemetry will likely be different that the post messages.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the repository's Python MQTT pub/sub implementation and identify how Sarracenia-style post messages are represented and routed. The issue does not name files, tests, or an acceptance criterion; clarify the telemetry format, routing destination, and separate-infrastructure design before implementation can be considered done.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, networking
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100