Azure / Azure/iotedge

Variable scrape frequency for different metrics

Open
#7,086 1 comment 0 reactions 0 assignees View on GitHub
customer-reported iotedge no-issue-activity
Dominant language
C#
Stars
1.5k
Forks
473
Avg merge
2d 17h
Merged PRs (30d)
7

Description

Hi!

Currently, we can control the drequency of scraping via ScrapeFrequencyInSecs.

However, we have different requirements for various metrics in terms of granularity.
https://github.com/Azure/iotedge/blob/main/edge-modules/metrics-collector/src/README.md

Practical example:
* Getting generic system information (Disk usage, etc) once per hour would be fine.
* For application-defined metrics we are happy with more fine-grained intervals.

Right now we need to use the lowest common value. This leads to increased costs in terms of network traffic, ingestions, and storage costs.
What we suggest is an extension of the behavior of BlockedMetrics towards a new element like "TimeFilteredMetrics". This would basically be a list of elements consisting of
* a time definition ala ScrapeFrequencyInSecs.
* A regex of metrics ala BlockedMetrics
This would block reporting of the included metrics *unless* at least the specified frequency has been passed.

Any thoughts on this, would this be useful? Would you be willing to accept a PR for such a change?

Note: I have also posted this at https://feedback.azure.com/d365community/idea/e929bf5b-fa40-ee11-a81c-000d3a040137.
But scrolling over that, it appears to have little to no activity.

Thanks and best regards,
Florian

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with edge-modules/metrics-collector/src/README.md and review the existing ScrapeFrequencyInSecs and BlockedMetrics behavior. Define how the proposed TimeFilteredMetrics entries combine a frequency with a metric regex, then confirm the expected configuration and filtering behavior with maintainers. Done means different metric groups can be reported at their configured intervals without using the lowest common frequency.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.