graphprotocol / graphprotocol/graph-node
[Bug] v0.40.1 - "graphman reassign" operations fails ~ 50% of the time without any meaningful info in subgraph's logs
Nadie ha tomado este issue todavía.
- Lenguaje dominante
- Rust
- Estrellas
- 3.2k
- Forks
- 1.1k
- Merge medio
- 4 d 1 h
- PR fusionados (30 d)
- 1
Descripción
Bug report
Hi Team,
I'm using grapnode v0.40.1. Roughly 50% of my "graphman reassign" operations look like below:
- Subgraph is running perfectly fine (healthy, no lag, not paused) on indexer node X
- I invoke "graphman reassign SUBGRAPH_HASH indexer_node_Y" command and it finalises without problems
- I check the status of the subgraph with "graphman info --status" command after few minutes and the subgraph is stuck, it isn't processing any new blocks
- Target index node doesn't produce logs for this subgraph
- Prometheus metrics (for example "deployment_head") are available on the previous index node and on the new/target indexer node, on the previous index node they have misleading values as processing doesn't really take place on it anymore
- I have to perform at least 1 more reassign operation to fix the subgraphs, sometimes up to 5 reassignment operations have to be done
I checked existing issues and it may be the case that https://github.com/graphprotocol/graph-node/issues/5253 is related so I've added a comment to it.
Can someone please take a look?
Relevant log output
IPFS hash
No response
Subgraph name or link to explorer
No response
Some information to help us out
- Tick this box if this bug is caused by a regression found in the latest release.
- Tick this box if this bug is specific to the hosted service.
- I have searched the issue tracker to make sure this issue is not a duplicate.
OS information
None
Guía de contribución
Primeros pasos
- Lee el issue completo y luego la guía de contribución del proyecto.
- Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
- Haz un fork del repositorio y trabaja en una rama.
- Abre un pull request que haga referencia al número del issue.
Línea de trabajo
Comienza reproduciendo el workflow intermitente de graphman reassign SUBGRAPH_HASH indexer_node_Y y, después, inspecciona graphman info --status, comparando los logs y deployment_head en ambos nodos indexer. Revisa el issue relacionado #5253 para obtener contexto. Se considera terminado cuando la reasignación comienza a procesarse de forma consistente en el nodo de destino, con métricas precisas y logs útiles cuando falla.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- rust
- Área
- cli, distributed-systems
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Activo
- Claridad
- Bastante claro
- Aptitud para principiantes
- 45/100