ChainSafe / ChainSafe/lodestar
Beacon Chain Harness For Testing
- Dominant language
- TypeScript
- Stars
- 1.4k
- Forks
- 483
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 156
Description
### Problem description
Currently when it comes to [testing](https://chainsafe.github.io/lodestar/contribution/testing/) the correctness of new changes that involve interacting with `BeaconChain` we rely on 1) unit tests 2) simulation tests 3) spec tests.
Unit tests require mocking up a chain, blocks and/or states. However since these objects contain many fields, one can only mock small parts of them that are related to the new changes. The nature of Ethereum protocol mandates validation in many places and a chain/block/state is only considered valid if the combination of the value in all fields pass the validation. Therefore passing unit tests with mocked chain/block/state does not guarantee the changes will work with a valid chain/block/state. Lodestar provides `MockedBeaconChain` but it only provides a chain that is TS type check correct and not protocol correct.
Simulation test creates an ephemeral devnet to test the overall functionality of Lodestar. It might be an overkill to test a small change. Also its error outputs might not be useful to tell which part of the new change is wrong. It is also resource intensive that is not designed to run on local machines for sanity checks but on the CIs.
Spec tests cover the correctness of the consensus protocol only. Things like caches, regen, Beacon api also interact with chain/block/state but not part of the protocol.
It is not hard to see the benefit of the ability to easily create a valid and functioning chain/block/state that is interact-able during unit tests.
### Solution description
Create a suite of test utilities that can
- Create a valid beacon chain that can
- Process blocks
- Create a valid genesis state
- Able to serve reasonably according to `IBeaconChain` interface
- Create a valid beacon block that can
- Go through state transition
- Be created with desired properties
- Contains proposer slashing
- Missed attestations from some validators
- Split view on head block
- Absent or present of sync committee
- etc.
- Create valid unaggregated attestations
- Create valid sync committee messages
- Create valid slashings
- Create valid withdrawals/deposits
- Create valid bls to execution change
- Dial state forward to certain slot
The test suite in general should be:
- Lightweight
- Verbal on logs
- Can be well inspected
### Additional context
Lighthouse's harness for reference: https://github.com/sigp/lighthouse/blob/b5bae6e7a2f52310617a3b93ab7579f3dc9d7880/beacon_node/beacon_chain/src/test_utils.rs#L604
Contributor guide
Research direction
Start with the testing documentation and the existing MockedBeaconChain, BeaconChain, and IBeaconChain references. Review Lighthouse's beacon_node/beacon_chain/src/test_utils.rs as the stated reference, then define the harness utilities around valid chains, blocks, states, and operations listed in the issue. Done means the suite is lightweight, inspectable, and supports the requested valid-chain testing scenarios.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- blockchain, testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100