graphprotocol / graphprotocol/graph-node
[Bug] blockHashFromNumber returns null after 10s when indexing permits are exhausted (0.43.0+)
Personne n'a encore pris cette issue.
- Langage dominant
- Rust
- Étoiles
- 3.2k
- Forks
- 1.1k
- Merge moyen
- 4 j 1 h
- PR mergées (30 j)
- 1
Description
Bug report
Since #6491 (0.43.0), Chain::block_pointer_from_number checks chain_store.block_ptrs_by_numbers before calling the RPC. If the block isn't in recent_blocks_cache, that lookup goes to blocks_from_store_by_numbers, which calls pool.get_permitted(). So it waits for the indexing semaphore (pool_size + GRAPH_EXTRA_QUERY_PERMITS), with no timeout.
The index-node resolver wraps the lookup in BLOCK_HASH_FROM_NUMBER_TIMEOUT (10s). When indexing holds every permit, blockHashFromNumber returns null and the RPC is never tried. 0.41.1 went straight to the RPC.
indexer-agent reports this as Failed to query graph node for blockHashFromNumber: no data returned (IE070), which blocks epoch resolution and POI closes. Epoch start blocks are hours old, so they always take the store path.
Repro: v0.45.0 against 0.41.1, pool_size = 2, mainnet and gnosis ingestors. To exhaust the permits I held LOCK TABLE public.ethereum_networks IN EXCLUSIVE MODE. Both ingestors then wait in the chain head UPDATE while holding their permit, which I confirmed in pg_stat_activity. It's a stand-in for write load.
| build | block 25961537 (not in recent cache) | recent block |
|---|---|---|
| 0.41.1 | hash, 0.08s | hash, 0.12s |
| 0.45.0 | null, 10.04s |
hash, 0.02s |
| 0.45.0 with the cache check removed | hash, 0.08s | hash, 0.08s |
store_semaphore_wait_ms doesn't show it. The wait is recorded only once the permit is acquired, and this lookup is cancelled before that (the metric read 52 during the null).
GRAPH_EXTRA_QUERY_PERMITS=4 does return the hash, in 5.11s, because the lookup times out on the pool connection and falls through to the RPC. That timeout calls state_tracker.mark_unavailable though, so it isn't a good workaround.
Suggested fix: drop the cache check from block_pointer_from_number, as #6537 did for is_on_main_chain, because the block cache isn't finality-aware here either. The callers are the index-node resolver, the registrar on deploy/graft and the public POI fallback, none of them hot. Still present on master (6838f4e3c).
Can open a PR with that if the direction sounds right?
Relevant log output
WARN Failed to fetch block hash from number, error: deadline has elapsed, block_number: 25961537, chain: ethereum, network: mainnet, component: IndexNodeServer > IndexNodeResolver
INFO Query timing (GraphQL), block: 0, query_time_ms: 10002, variables: null, query: { version { version } blockHashFromNumber(network: "mainnet", blockNumber: 25961537) } , subgraph_id: indexnode, component: IndexNodeServer
IPFS hash
No response
Subgraph name or link to explorer
No response
Some information to help us out
- Tick this box if this bug is caused by a regression found in the latest release.
- Tick this box if this bug is specific to the hosted service.
- I have searched the issue tracker to make sure this issue is not a duplicate.
OS information
macOS
Guide de contribution
Ouvrir le guide de contribution
Par où commencer
- Lisez l'issue en entier, puis le guide de contribution du projet.
- Signalez en commentaire que vous la prenez — cela évite que deux personnes fassent le même travail.
- Forkez le dépôt et travaillez sur une branche.
- Ouvrez une pull request qui référence le numéro de l'issue.
Piste de recherche
Commencez à Chain::block_pointer_from_number et suivez chain_store.block_ptrs_by_numbers jusqu’à blocks_from_store_by_numbers, puis examinez les appelants du résolveur de nœuds d’index. Reproduisez le problème avec les permis d’indexation épuisés et vérifiez que blockHashFromNumber atteint le RPC et renvoie le hash dans le délai d’attente existant, sans marquer l’état comme indisponible ; ajoutez une couverture de régression si un test pertinent existe.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- postgresql, rust
- Domaine
- backend, blockchain, databases
- Type d'issue
- Bug
- Difficulté
- 3/5
- Temps estimé
- 1-2 jours
- Activité
- Active
- Clarté
- Clairement spécifiée
- Accessibilité débutants
- 74/100