Ingest historical block hashes

Open
#3,518 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
25/100
Issue type
Feature
Clarity
Needs clarification
Activity status
Stale
Tech stack
blockchain, rust

Research direction

Start by reviewing the block ingestor and block cache behavior described in the issue, then compare how the firehose and JSON-RPC paths provide historical block data. Done means historical block hashes and numbers can be queried reliably, including hashes missed by the normal reorg-threshold ingestion.

Written by the indexing model from the issue text.

Description

area/indexing Stale

An issue affecting the network, which sends queries by block hash, is that graph node will fail if it cannot find the block hash in its block cache. The block ingestor will ingest blocks within the reorg threshold, but holes can happen and nothing is done to fill them.

Since historical queries mean that all block hashes are potentially necessary, a way to solve this is to run a backfill ingestor job that ingests all historical hashes.

In terms of implementation, we'd want to be able to have only the block hash and number rather than the whole header, which is not valid in the current block cache structure. Also we'd probably want to leverage the firehose and not need a json-rpc implementation of the backfill.

Dominant language
Rust
Stars
3.2k
Forks
1.1k
Avg merge
4d 1h
Merged PRs (30d)
1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from graphprotocol/graph-node

All issues in graphprotocol/graph-node

Similar issues

More Rust issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.