IntersectMBO / IntersectMBO/cardano-ledger
New versioned serialization API for data types that don't live on chain.
- Dominant language
- Haskell
- Stars
- 295
- Forks
- 179
- Avg merge
- 4d 7h
- Merged PRs (30d)
- 29
Description
Recently in #3063 and #3138 we introduced versioned serialization mechanism that is necessary for data types which live on chain. Here are the reasons why that versioning scheme is perfect for types that get placed on chain (eg. `TxBody` , `TxAuxData`, etc.):
* It is only natural to switch serialization changes together with all other changes at a hard fork boundary, therefore major protocol version, which is bumped at that boundary, can be used perfectly for versioning serialization
* Deserialization of every type must be backwards compatible at least with the previous era, so there is no need for translation logic, from older version to the newer.
* Every type that lives on chain is parameterized on `era`, therefore it is always possible to get the protocol version for deserialization if necessary. This, however, comes with a caviat that we can't introduce any serialization changes for intra-era hard forks, because as of now we don't know the exact protocol version a the type level, we only know the range.
The other end of the spectrum are the types that do not live on chain, for example most of the contents of the ledger state (i.e. `NewEpochState`) is only stored locally on each node. Such type require a more classical serialization versioning scheme. In such scheme the version that the type was serialized with is encoded in the binary blob together with the data itself, usually in some sort of header. We want to use such scheme because we want to be able to upgrade those local types at a node software version bump, rather than at a hardfork boundary. Due to that fact protocol version is not a suitable candidate. So we need a versioning scheme that:
* Encode a version together with the data. It should probably not be done at the type level (i.e. every individual type is prefixed with a version), because that would introduce quite a bit overhead and is just an overkill.)
* Every type would provide an upgrade logic for going from the previous version of a type to the newer one, whenever there was a breaking change to deserialization.
* Deserialization interface should have native support for improving memory sharing. There is already an implementation of such interface in `FromSharedCBOR`.
Contributor guide
Assessment
This issue has not been assessed yet.