prometheus / prometheus/client_python

Remove gauge metric

未关闭
#1,055 7 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

主要语言
Python
星标
4.4k
派生
876
平均合并
8 天 4 小时
30 天内合并 PR
1

描述

Hi all,
I have written a custom exporter for calculating the cost of using a node in AWS.
Here is how it works:
Lets assume I have 3 nodes, each costs 5$/ 1h, when I plot using grafana the sum(cost_metric{}) i get 15$ (3x5$). Lets say after 2 hours one of the nodes get deleted (autoscaling). In that case the total cost should drop to 10$.

The problem is that in my case the metric is preserved and even though the node has been deleted the cost is kept and thus it displays 15$ instead of dropping to 10$

How would I go about saving that problem?

cost_metric = Gauge(
    "cost_metric ",
    "Cost of running an instance for 1 hour",
    ["node_name", "instance_type"],
)
...

node_names = get_nodes()
    for node_name in node_names:
        node_info = get_node_info(node_name)
        if node_info is None:
            continue

        logging.info(f"Updating metrics for node: {node_name}")

        # labels section
        labels = node_info["metadata"]["labels"]
        instance_type = labels.get("beta.kubernetes.io/instance-type", "unknown")
        cost = get_cost_of_instance(instance_type)

        if cost is not None:
            cost_metric.labels(node_name=node_name, instance_type=instance_type).set(cost)

I tried

I collected previous and current nodes in form of dict and then wanted to removed the ones that arent existing, the issue is that :

cost_metric.remove(node_name=node_name, instance_type=instance_type)

Traceback (most recent call last):
  File "/home/XXXX/projects/main.py", line 154, in <module>
    previous_nodes = update_metrics(previous_nodes)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/eksohio/projects/main.py", line 131, in update_metrics
    cost.remove(node_name=node_name, instance_type=instance_type)
TypeError: MetricWrapperBase.remove() got an unexpected keyword argument 'node_name'

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

payload 在 traceback 中指出了 Gauge 声明和 MetricWrapperBase.remove 调用,但没有指出任何 repository 文件或测试。首先检查 Gauge 和 MetricWrapperBase 用于移除 label 的 API,并审查所示的 exporter 循环。完成的标准是确定移除已删除节点的 metric 的受支持方式,或确认需要修改库。

由索引模型根据 Issue 内容生成。

评估

技术栈
aws, kubernetes, prometheus, python
领域
cloud, infrastructure, observability
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
需要澄清
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。