IntersectMBO / IntersectMBO/ouroboros-consensus

Improve golden tests infrastructure

Open
#694 5 comments 0 reactions 0 assignees View on GitHub
better-tests
Dominant language
Haskell
Stars
67
Forks
43
Avg merge
5d 13h
Merged PRs (30d)
43

Description

At the moment, we have a manually constructed examples, leading to the possibility of lots of missed parts of the encoding, and not all tests live where they ought to.

We should:

1. Introduce a generic infrastructure (probably in `cardano-base`) for constructing canonical examples:

```haskell
class CanonicalExamples a where
canonicalExamples :: Proxy a -> [a]
```

which should have a default generic implementation: for a sum type just construct canonical examples for each of the constructors, and for a product type just construct a single canonical example for all components.

2. Manual instances should be provided for (a) recursive datatypes (the above strategy would otherwise give an infinite number of examples), (b) the standard types (tuples, maybe, lists, ..), and (c) types where the decoder establishes specific invariants. As an example of category (c), suppose we have a data type with three fields `a`, `b`, and `c`, where the encoder only encodes `a` and `b`, and the coder reconstructs `c` from `a` and `b`. An arbitrary example would fail a roundtrip test, as the two `c` values would not match, and so this will require a manual instance. Writing such a manual instance can piggy back on canonical examples for standard types (i.e., it could just ask for a canonical example of an `a`, `b` tuple).

3. Instances should be provided for all the ledger types, in `cardano-ledger` for Byron and in `cardano-ledger-specs` for Shelley, and these repos should be equipped with golden tests for these canonical examples.

4. In `ouroboros-consensus`, we should provide instances for _our_ types, not duplicating any of the work done in the ledger, and have golden tests for our types. Some of those tests will necessarily involve types from the ledger (i.e., `ExtLedgerState`), that is not problematic.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing manually constructed examples and golden tests across cardano-base, cardano-ledger, cardano-ledger-specs, and ouroboros-consensus. Define the shared canonical-example infrastructure, add the required manual and ledger or consensus instances without duplication, and ensure golden tests cover the canonical examples in each repository.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
blockchain, distributed-systems, testing
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.