graphprotocol / graphprotocol/graph-node

[Feature] Add subgraph name and version as label to prometheus metrics

Open
#5,278 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement Stale
Dominant language
Rust
Stars
3.2k
Forks
1.1k
Avg merge
4d 1h
Merged PRs (30d)
1

Description

Description

Currently, I check for the status of my subgraph nodes with the following prometheus query:
(ethereum_chain_head_number{} - ignoring(deployment, job, network, shard) deployment_head{} > 100) and ignoring(deployment, job, network, shard)(deriv(deployment_head{}[10m]) * 60 <= 0)

In simple terms, the query checks for the differences between the ethereum_chain_head_number metric and the deployment_head metric. If it is more than 100 blocks AND there hasn't been any significant changes in the deployment_head value over a 10 minutes window, the node is flagged as out of sync. The two metrics do not have the same labels, hence the ignoring of labels as seen in the query.

This query works fine for nodes hosting one or more subgraphs of different networks. However, when a subgraph of the same network but different subgraphName is deployed to a node, the only way to differentiate them is through the deployment label, which is a hash number that changes for any update and therefore, complex to manage. This introduces an issue as deployment_head starts coming off as a duplicate metric.

sample:

Assuming the two metrics below are for two subgraphs (sepolia-1 and sepolia-2), I can't tell them apart, and as far as I don't filter by deployment, it is a duplicate metric when evaluating it.

deployment_head{deployment=<HASH IPFS deployment number 1>, instance=<node_url>, job="mymetrics", network="sepolia", shard="primary"}

deployment_head{deployment=<HASH IPFS deployment number 2>, instance=<node_url>, job="mymetrics", network="sepolia", shard="primary"}
Proposal:

Add subgraphName and subgraphVersion to the BlockStreamMetrics struct. subgraphName being top priority.

deployment_head{deployment=<HASH IPFS deployment number 1>, instance=<node_url>, job="mymetrics", network="sepolia", shard="primary", subgraph="sepolia-1", version="v0.1.0"}

deployment_head{deployment=<HASH IPFS deployment number 2>, instance=<node_url>, job="mymetrics", network="sepolia", shard="primary", subgraph="sepolia-2", version="v0.1.0"}
Are you aware of any blockers that must be resolved before implementing this feature? If so, which? Link to any relevant GitHub issues.

No response

Some information to help us out
  • Tick this box if you plan on implementing this feature yourself.
  • I have searched the issue tracker to make sure this issue is not a duplicate.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Locate the BlockStreamMetrics struct and trace where its fields become Prometheus metric labels. Start by checking the existing deployment_head metric path and related tests, if present. Done means the metrics expose subgraph name and version labels, with subgraph name prioritized as proposed.

Written by the indexing model from the issue text.

Assessment

Tech stack
prometheus, rust
Domain
observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.