IntersectMBO / IntersectMBO/cardano-ledger

Implementation-Independent Ledger Conformance Test Suite

Open
#4,892 23 comments 3 reactions 0 assignees View on GitHub
Dominant language
Haskell
Stars
295
Forks
179
Avg merge
4d 7h
Merged PRs (30d)
29

Description

## Motivation
The availability of [Plutus Conformance Test Suite](https://github.com/IntersectMBO/plutus/tree/master/plutus-conformance) has fostered the development of numerous Plutus implementations in [Rust](https://github.com/pragma-org/uplc), [Python](https://github.com/OpShin/uplc), [JavaScript](https://github.com/HarmonicLabs/uplc), [Go](https://github.com/blinklabs-io/plutigo), and [C++](https://github.com/sierkov/daedalus-turbo/tree/main/lib/dt/plutus). These implementations support diverse community use cases and enable experimentation with various aspects of Plutus, including performance.

The community is actively working on Cardano Ledger implementations in [Rust](https://github.com/txpipe/pallas), [Go](https://github.com/blinklabs-io/gouroboros/tree/main), and [C++](https://github.com/sierkov/daedalus-turbo). However, without a high-quality, implementation-independent test suite like the one available for Plutus, each implementation must rely on its own custom tests. This increases the risk of non-conformant implementations being actively used simply due to the lack of a universally available conformance test suite. Furthermore, a language-independent test suite would facilitate collaboration, allowing contributions from any team to benefit all implementations.

The task consists of two stages:
- Alignment: Discuss and define the requirements and general approach for an implementation-independent test suite.
- Development: Build the test suite, with the [the C++ implementation](https://github.com/sierkov/daedalus-turbo) committing a share of the necessary resources to support its development.

More details follow. Feedback and suggestions are welcome.

## Requirements
- The test suite must be language- and environment-agnostic.
- It must be reasonably simple to use in major programming languages.
- It must follow a black-box approach, evaluating outputs based on given inputs, to enable experimentation with alternative algorithms such as batching and parallelization.
- It should support parallel execution of individual tests to leverage modern hardware and shorten development cycles.

## Assumptions Driving the Proposed Approach
The initial proposed approach is based on the following assumptions:
- Every ledger implementation requires a functional CBOR decoder and corresponding parsers for Cardano block formats.
- A ledger implementation can be fully modeled as a black box that processes a sequence of blocks and produces a new ledger state.
- A common approach to testing a new ledger implementation is to process blocks from Cardano’s mainnet and testnets. The results are then compared against those produced by the reference implementation (Cardano Node). Therefore, it should be possible to leverage some of those data when creating the test suite.
- The formal specification of the Cardano Ledger serves as the authoritative source for ledger rules and should be used to generate a comprehensive set of test cases. However, automatic source code generation from the formal specification is limited to only a few programming languages, thereby affecting the test suite’s primary objective: implementation independence.

## Proposed approach
- Test case inputs should use the standard CBOR format used for storing blocks on the Cardano mainnet.
- Test case outputs should use the state snapshot format from the latest stable version of the Cardano Node, which is also CBOR-encoded. This allows for easy benchmarking against the reference implementation. Additionally, if the snapshot format changes, regenerating outputs is straightforward, as test case inputs remain standard Cardano blocks.
- Each test case consists of an initial ledger state (including the genesis configuration), a sequence of blocks, and a final ledger state.
- Initial test cases can be created by sampling blocks from the Cardano mainnet and modifying them as needed to ensure consistency of generated sequences with ledger specification rules.
- A comprehensive test set can be generated either programmatically, using a block generator based on the formal ledger specification, or manually, depending on cost-effectiveness.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.