[Bug] graph-node temporary outage with `store error: NotFound`

Open
#4,739 14 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
25/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
postgresql, rust
Domain
backend, databases

Research direction

Start with the reported Block stream error and the SubgraphInstanceManager component, then trace how the store returns NotFound during synchronization. Use the supplied log context and PostgreSQL/graph-node version details to reproduce or isolate the temporary stop across subgraphs; done means identifying the cause and verifying that synchronization recovers without the error.

Written by the indexing model from the issue text.

Description

bug Stale
Bug report

We experience an error (logs attached) while running the subgraph causing a temporary sync stop:
image

Seems like the error is linked to some store problem but we can't find any problems. All metrics are good for both PostgreSQL and graph-node.

99% of the time node is working absolutely fine.
Also, this problem occurs on all subgraphs at the same time.

Graph version is v0.31.0
Docker image is graphprotocol/graph-node:v0.31.0

Relevant log output
Jul 04 08:23:51.258 DEBG Block stream produced a non-fatal error, error: store error: NotFound, sgd: 319, subgraph_id: QmQy5znQNDrZiVq8L7o3zFaTadiQz3tzgfXASBLz2WFgXL, component: SubgraphInstanceManager
IPFS hash

No response

Subgraph name or link to explorer

No response

Some information to help us out
  • Tick this box if this bug is caused by a regression found in the latest release.
  • Tick this box if this bug is specific to the hosted service.
  • I have searched the issue tracker to make sure this issue is not a duplicate.
OS information

None

Dominant language
Rust
Stars
3.2k
Forks
1.1k
Avg merge
4d 1h
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from graphprotocol/graph-node

All issues in graphprotocol/graph-node

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.