Variable scrape frequency for different metrics
- Dominant language
- C#
- Stars
- 1.5k
- Forks
- 473
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 7
Description
Hi!
Currently, we can control the drequency of scraping via ScrapeFrequencyInSecs.
However, we have different requirements for various metrics in terms of granularity.
https://github.com/Azure/iotedge/blob/main/edge-modules/metrics-collector/src/README.md
Practical example:
* Getting generic system information (Disk usage, etc) once per hour would be fine.
* For application-defined metrics we are happy with more fine-grained intervals.
Right now we need to use the lowest common value. This leads to increased costs in terms of network traffic, ingestions, and storage costs.
What we suggest is an extension of the behavior of BlockedMetrics towards a new element like "TimeFilteredMetrics". This would basically be a list of elements consisting of
* a time definition ala ScrapeFrequencyInSecs.
* A regex of metrics ala BlockedMetrics
This would block reporting of the included metrics *unless* at least the specified frequency has been passed.
Any thoughts on this, would this be useful? Would you be willing to accept a PR for such a change?
Note: I have also posted this at https://feedback.azure.com/d365community/idea/e929bf5b-fa40-ee11-a81c-000d3a040137.
But scrolling over that, it appears to have little to no activity.
Thanks and best regards,
Florian
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with edge-modules/metrics-collector/src/README.md and review the existing ScrapeFrequencyInSecs and BlockedMetrics behavior. Define how the proposed TimeFilteredMetrics entries combine a frequency with a metric regex, then confirm the expected configuration and filtering behavior with maintainers. Done means different metric groups can be reported at their configured intervals without using the lowest common frequency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100