[Bug] Rewinding a subgraph causes a constraint violation in graph-node that in turn causes indexer-agent to crashloop
还没有人认领这个 Issue。
评估
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 新手友好度
- 35/100
- Issue 类型
- 缺陷
- 描述清晰度
- 需要澄清
- 活跃度
- 停滞
- 技术栈
- postgresql, rust
调研方向
首先检查 graph-node 和 indexer-agent 的日志输出,尤其是 packages/indexer-common/dist/graph-node.js 和 packages/indexer-common/dist/errors.js,然后调查 entityCount 约束相关的 rewind 和索引状态路径。重现 graphman 的 rewind 或 drop 场景,并确认 graph-node 恢复同步且 indexer-agent 不再进入 crashloop。
由索引模型根据 Issue 内容生成。
描述
Bug report
graph-node:v0.34.1
indexer-agent:v0.20.22
Activities that were undertaken before observing this bug:
- Cleared call_cache for Arbitrum as part of a complex subgraph sync performance troubleshooting exercise via psql
- Rewound a specific problematic subgraph, Silo Finance Arbitrum,
QmTMKqty5yZvZtB3SwzXUG92aZUH1YQw3VjByGw4wgaMhWto block 1 usinggraphman - Observed the above subgraph syncing to ~130m blocks, then stalled.
- Checked graph-node logs and found related error (see log output)
- Observed indexer-agent complaining about same issue and crash looping - cannot use the agent at all right now to manage subgraphs (see log output)
IMPACT: Production Indexer at risk; we cannot manage our online and offline allocations while we have this issue - ideally need a temp fix for the specific symptoms. Would graphman drop resolve the issue? Would the graph-node and indexer-agent be able to handle that and start syncing the sub again from scratch given this is a subgraph in flight with live allocations?
Relevant log output
----- GRAPH-NODE
Apr 04 13:24:34.037 ERRO Subgraph instance failed to run: internal constraint violated: Subgraph writer for QmTMKqty5yZvZtB3SwzXUG92aZUH1YQw3VjByGw4wgaMhW[sgd622] is not running, sgd: 622, subgraph_id: QmTMKqty5yZvZtB3SwzXUG92aZUH1YQw3VjByGw4wgaMhW, component: SubgraphInstanceManager
Apr 04 13:48:18.741 WARN Price provider Removed: 0x8dca64a43865454f41aa1a3cf0140eb89f2c08aa53871235ecbe46b6a309a1e3, data_source: PriceProvidersRepository, sgd: 622, subgraph_id: QmTMKqty5yZvZtB3SwzXUG92aZUH1YQw3VjByGw4wgaMhW, component: SubgraphInstanceManager > UserMapping
Apr 04 13:48:18.742 ERRO Oracle was not found when trying to remove it at txn: 0x8dca64a43865454f41aa1a3cf0140eb89f2c08aa53871235ecbe46b6a309a1e3, data_source: PriceProvidersRepository, sgd: 622, subgraph_id: QmTMKqty5yZvZtB3SwzXUG92aZUH1YQw3VjByGw4wgaMhW, component: SubgraphInstanceManager > UserMapping
----- INDEXER-AGENT
{"level":50,"time":1712241706593,"pid":1,"hostname":"268ad9e1400b","name":"IndexerAgent","component":"GraphNode","err":{"type":"IndexerError","message":"Failed to query indexing status API","stack":"IndexerError: Failed to query indexing status API\n at indexerError (/opt/indexer/packages/indexer-common/dist/errors.js:173:12)\n at GraphNode.<anonymous> (/opt/indexer/packages/indexer-common/dist/graph-node.js:146:55)\n at Generator.next (<anonymous>)\n at fulfilled (/opt/indexer/packages/indexer-common/dist/graph-node.js:5:58)\n at processTicksAndRejections (node:internal/process/task_queues:96:5)","code":"IE018","explanation":"https://github.com/graphprotocol/indexer/blob/main/docs/errors.md#ie018","cause":{"type":"CombinedError","message":"[GraphQL] Store error: internal constraint violated: the entityCount for QmTMKqty5yZvZtB3SwzXUG92aZUH1YQw3VjByGw4wgaMhW is not representable as a u64","name":"CombinedError","graphQLErrors":[{"message":"Store error: internal constraint violated: the entityCount for QmTMKqty5yZvZtB3SwzXUG92aZUH1YQw3VjByGw4wgaMhW is not representable as a u64"}],"response":{"size":0,"timeout":0}}},"msg":"Failed to query indexing status API"}
IPFS hash
No response
Subgraph name or link to explorer
Some information to help us out
- Tick this box if this bug is caused by a regression found in the latest release.
- Tick this box if this bug is specific to the hosted service.
- I have searched the issue tracker to make sure this issue is not a duplicate.
OS information
Linux
- 主要语言
- Rust
- 星标
- 3.2k
- 派生
- 1.1k
- 平均合并
- 4 天 1 小时
- 30 天内合并 PR
- 1
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
graphprotocol/graph-node 的其他 Issue
-
current: include emits an all-null bucket for dimensionless aggregations, nulling the whole response 未关闭
难度 2/5 1-3 小时 新手友好度 78/100
graphprotocol/graph-node#6719 ·
-
难度 2/5 1-3 小时 新手友好度 68/100
graphprotocol/graph-node#6673 ·
-
难度 2/5 1-3 小时 新手友好度 70/100
graphprotocol/graph-node#6650 · 1 条评论 ·
-
难度 4/5 3-5 天 新手友好度 48/100
graphprotocol/graph-node#6722 ·
-
难度 3/5 1-2 天 新手友好度 68/100
graphprotocol/graph-node#6721 ·
查看 graphprotocol/graph-node 的全部 Issue
相似的 Issue
-
risk:low runtime status:in-progress type:test
难度 1/5 1 小时以内 新手友好度 92/100
zeroclaw-labs/zeroclaw#11023 ·
-
good first issue refactor
难度 2/5 1-3 小时 新手友好度 72/100
-
难度 2/5 1-3 小时 新手友好度 86/100
kwakseongjae/auto-hwp#319 ·
-
area:cli bug filter-quality good first issue priority:medium
难度 2/5 1-3 小时 新手友好度 84/100
-
难度 1/5 1 小时以内 新手友好度 72/100
bevyengine/bevy#25861 ·