graphprotocol / graphprotocol/graph-node
[Bug] blockHashFromNumber returns null after 10s when indexing permits are exhausted (0.43.0+)
Nadie ha tomado este issue todavía.
- Lenguaje dominante
- Rust
- Estrellas
- 3.2k
- Forks
- 1.1k
- Merge medio
- 4 d 1 h
- PR fusionados (30 d)
- 1
Descripción
Bug report
Since #6491 (0.43.0), Chain::block_pointer_from_number checks chain_store.block_ptrs_by_numbers before calling the RPC. If the block isn't in recent_blocks_cache, that lookup goes to blocks_from_store_by_numbers, which calls pool.get_permitted(). So it waits for the indexing semaphore (pool_size + GRAPH_EXTRA_QUERY_PERMITS), with no timeout.
The index-node resolver wraps the lookup in BLOCK_HASH_FROM_NUMBER_TIMEOUT (10s). When indexing holds every permit, blockHashFromNumber returns null and the RPC is never tried. 0.41.1 went straight to the RPC.
indexer-agent reports this as Failed to query graph node for blockHashFromNumber: no data returned (IE070), which blocks epoch resolution and POI closes. Epoch start blocks are hours old, so they always take the store path.
Repro: v0.45.0 against 0.41.1, pool_size = 2, mainnet and gnosis ingestors. To exhaust the permits I held LOCK TABLE public.ethereum_networks IN EXCLUSIVE MODE. Both ingestors then wait in the chain head UPDATE while holding their permit, which I confirmed in pg_stat_activity. It's a stand-in for write load.
| build | block 25961537 (not in recent cache) | recent block |
|---|---|---|
| 0.41.1 | hash, 0.08s | hash, 0.12s |
| 0.45.0 | null, 10.04s |
hash, 0.02s |
| 0.45.0 with the cache check removed | hash, 0.08s | hash, 0.08s |
store_semaphore_wait_ms doesn't show it. The wait is recorded only once the permit is acquired, and this lookup is cancelled before that (the metric read 52 during the null).
GRAPH_EXTRA_QUERY_PERMITS=4 does return the hash, in 5.11s, because the lookup times out on the pool connection and falls through to the RPC. That timeout calls state_tracker.mark_unavailable though, so it isn't a good workaround.
Suggested fix: drop the cache check from block_pointer_from_number, as #6537 did for is_on_main_chain, because the block cache isn't finality-aware here either. The callers are the index-node resolver, the registrar on deploy/graft and the public POI fallback, none of them hot. Still present on master (6838f4e3c).
Can open a PR with that if the direction sounds right?
Relevant log output
WARN Failed to fetch block hash from number, error: deadline has elapsed, block_number: 25961537, chain: ethereum, network: mainnet, component: IndexNodeServer > IndexNodeResolver
INFO Query timing (GraphQL), block: 0, query_time_ms: 10002, variables: null, query: { version { version } blockHashFromNumber(network: "mainnet", blockNumber: 25961537) } , subgraph_id: indexnode, component: IndexNodeServer
IPFS hash
No response
Subgraph name or link to explorer
No response
Some information to help us out
- Tick this box if this bug is caused by a regression found in the latest release.
- Tick this box if this bug is specific to the hosted service.
- I have searched the issue tracker to make sure this issue is not a duplicate.
OS information
macOS
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Línea de trabajo
Comienza en Chain::block_pointer_from_number y sigue chain_store.block_ptrs_by_numbers hasta blocks_from_store_by_numbers; luego inspecciona los llamadores del resolvedor de nodos de índice. Reproduce el problema con los permisos de indexación agotados y verifica que blockHashFromNumber llegue al RPC y devuelva el hash dentro del timeout existente, sin marcar el estado como no disponible; añade cobertura de regresión si existe una prueba relevante.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- postgresql, rust
- Área
- backend, blockchain, databases
- Tipo de issue
- Error
- Dificultad
- 3/5
- Tiempo estimado
- 1-2 días
- Estado de actividad
- Activo
- Claridad
- Bien especificado
- Aptitud para principiantes
- 74/100