prometheus / prometheus/client_python
Aggregate expired pid db files, control the number of files and improve scrape effectiveness
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 4.4k
- 派生
- 876
- 平均合并
- 8 天 4 小时
- 30 天内合并 PR
- 1
描述
history related issues: https://github.com/prometheus/client_python/pull/441
https://github.com/prometheus/client_python/pull/430
Can we aggregate all the db files from a period of time ago and non-current pid into a total db file, to control the number of pid files ?
I have realized this idea with golang,here are some details:
Project deploy info:
gunicorn django
128 workers
gunicorn max_requests:10000(create a new pid file almost every minute)
- I can't solve the problem that the pid file has been growing, and it can reach 6,000 in four days;
- Try to delete the expired pid regularly in the code, but it will cause the figure to drop with grafana;
- The time to request metric is getting longer as the program runs.
Improve scrape efficiency:
I used golang to rewrite the logic of python aggregate metrics(generate metric still using python). After rewriting, each scrape time is less than 1 second.
Solve the growing pid files:
Aggregate all the db files from a period of time ago and non-current pid into a total db file. Then delete these files. When calculating metric, history total db + curent pid = current pid db. (i do it every hour)
Now, num of pid files is <200 in my project. if we can do change this, it would be a big strengthen. Just like prometheus will also aggregate historical data
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
先检查与历史相关的 pull request #441 和 #430,然后跟踪为 128 worker 的 Gunicorn 部署所描述的 Python 聚合和 scrape 行为。将提议的定期聚合与删除方法与 Grafana 中当前的指标结果进行比较;完成的标准是 pid 数据库文件大小有界、历史值得以保留,并且 scrape 时间低于一秒。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- django, go, grafana, prometheus, python
- 领域
- backend, observability, performance
- Issue 类型
- 功能
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 25/100