graphprotocol / graphprotocol/graph-node

[Feature] info.subgraph_sizes in sharded namespace view

Open
#6,038 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement ops Stale
Dominant language
Rust
Stars
3.2k
Forks
1.1k
Avg merge
4d 1h
Merged PRs (30d)
1

Description

Description

Firstly thank you for the new sharded namespace, we are implementing new grafana dashboards using the views. We have a general health dashboard that we have adapted to work with the sharded namespace views (added to end of this post).

The query works apart from the subgraph size field. I believe this is in the info schema in each subgraph shard.

Would it be possible to bring this data into the new sharded namespace views to complete this overview of subgraph health?

SELECT DISTINCT ON (sd.deployment)
    ds.name as deployment_schema,
    ds.shard as deployment_shard,
    ds.created_at as deployment_created_at,
    sg.name as deployment_name,
    sd.deployment,
    substring(sm.repository, 20, 31) as git_repo,
    sda.node_id,
    sd.health as sub_health,
    sd.failed as sub_failed,
    sd.synced_at as sub_synced_at,
    sd.latest_ethereum_block_number as last_block,
    sd.entity_count as sub_entity,
    size.total_bytes as size_total, 
    size.total_bytes as size_total_bytes, 
    sm.start_block_number as start_block, 
    encode(sm.start_block_hash, 'hex') as start_hash, 
    encode(sd.latest_ethereum_block_hash, 'hex') as last_hash,
    sd.reorg_count,
    sd.current_reorg_depth as curr_reorg_count,
    sd.max_reorg_depth as max_reorg,
    ds.network,
    (network.head_block_number - sd.latest_ethereum_block_number) AS blocks_behind_network,
    CASE 
        WHEN network.name = 'mainnet' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 5)::text
        WHEN network.name = 'rinkeby' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 4)::text
        WHEN network.name = 'kovan' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 15)::text
        WHEN network.name = 'poa-core' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 12)::text
        WHEN network.name = 'ropsten' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 2)::text
        WHEN network.name = 'arbitrum-one' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 230)::text
        WHEN network.name = 'arbitrum-sepolia' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 230)::text
        WHEN network.name = 'matic' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 30)::text
        WHEN network.name = 'base' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 30)::text
        WHEN network.name = 'optimism' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 30)::text
        WHEN network.name = 'avalanche' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 30)::text
        WHEN network.name = 'fantom' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 40)::text
        WHEN network.name = 'celo' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 12)::text
        WHEN network.name = 'gnosis' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 12)::text
        WHEN network.name = 'bsc' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 20)::text
        WHEN network.name = 'linea' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 30)::text
        WHEN network.name = 'scroll' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 20)::text
        WHEN network.name = 'boba' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 30)::text
        WHEN network.name = 'boba-bnb' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 5)::text
        WHEN network.name = 'moonbeam' THEN ((network.head_block_number - sd.latest_ethereum_block_number) / 10)::text
        ELSE 'N/A' 
    END AS lag
FROM sharded.subgraph_deployment AS sd
JOIN sharded.subgraph_manifest AS sm ON sm.id = sd.id
JOIN deployment_schemas AS ds ON ds.subgraph = sd.deployment
JOIN sharded.subgraph_version AS sv ON sv.deployment = sd.deployment
JOIN sharded.subgraph AS sg ON sv.id = sg.current_version -- use only current version to dedupe
LEFT JOIN sharded.subgraph_deployment_assignment AS sda ON sda.id = sd.id
LEFT JOIN info.subgraph_sizes AS size ON ds.name = size.name
JOIN sharded.ethereum_networks AS network ON network.name = ds.network
ORDER BY sd.deployment;```



### Are you aware of any blockers that must be resolved before implementing this feature? If so, which? Link to any relevant GitHub issues.

_No response_

### Some information to help us out

- [ ] Tick this box if you plan on implementing this feature yourself.
- [x] I have searched the issue tracker to make sure this issue is not a duplicate.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing how the sharded namespace views are defined and how info.subgraph_sizes is populated, using the SQL query in the issue as the expected shape. Compare the available sharded view columns with the size data, then verify that the requested subgraph size fields are queryable through the sharded namespace views.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.