IntersectMBO / IntersectMBO/ouroboros-consensus

Avoid header validation during block validation

Open
#657 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
67
Forks
43
Avg merge
5d 13h
Merged PRs (30d)
43

Description

We recently introduced the concept of header revalidation to mitigate Shelley's slow header validation: input-output-hk/ouroboros-network#2521.

Currently, we first download headers via the ChainSync protocol, validate them (expensive), the BlockFetch client then downloads the blocks corresponding to these headers, adds them to the ChainDB, which validates these blocks, *including their headers*. This means we revalidate each header *twice*, which is redundant and, worse, in the critical path of syncing.

We can avoid this by letting `ChainDB.addBlock` use `revalidateHeader` instead of `validateHeader` (some refactoring is probably required to make this work). `ChainDB.addBlock` should have as precondition that the header of the block is valid, which will be the case for all blocks added by the BlockFetch client.

However, we do add blocks of which we *haven't validated the headers yet* in one place, i.e., when adding a self-forged block. We can provide a variant of `ChainDB.addBlock` for this.

---

A rejected idea I had to avoid this distinction: introduce a separate cache of previously validated header hashes, similar to the cache of previously validated block hashes. It would be maintained by the ChainDB and the ChainSync client would add hashes to it. When the hash is already in the validated block cache, it doesn't have to be added to the validated header cache. When a block passes validation, its hash can be removed from the validated header cache and insert in the validated block cache. The validated header cache would be GC'ed similarly to how the validated block cache is GC'ed.

---

An idea @dcoutts had: if we have a previously validated header cache, we can use it to avoid validating the same header by different ChainSync threads. However, we have to be careful:

> If I have 20 upstream peers, and normally I scale the CPU use for validating 1 header (and reusing it 19 times) then when my peers attack me I now have a 20x CPU use spike, and I don't have the resources. And this is all under the control of the attacker.

Duncan proposes that we only do this optimisation when bulk syncing. This is a separate optimisation that we can consider later.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.