History expiry

Long term solution to the growing state problem.

Context

As we are getting further away from the genesis, keeping all historical state becomes impractical, thus nodes (especially validators) start pruning (https://github.com/oasisprotocol/docs/pull/1526).

However, as the nodes start pruning we are risking data availability issues (at least for the genesis sync).

Currently, the network still has nodes that serve historical data and we also offer snapshots. Nevertheless, there will be less and less such nodes, which decreases data availability and increases the risk of (accidentally) loosing it.

Proposed solution

History Expiry.

Let's define a sane default, for which all network participants (except for stateless clients) guarantee, it is possible to sycn from. I believe 1 year would be a good value.

If the network is always capable of serving a State DB checkpoint that is one year old, and if by default nodes always preserve 1 year of blocks and or storage diffs, then our network guarantees history expiry invariant.

Implementation space:

History Expiry Checkpoint

In addition to existing checkpoints, there will be another historical checkpoint created and replaced every month (target age of history expiry). In practice it will replace the current genesis checkpoint. I.e. every node serving checkpoints will fetch, store and serve it, even if they only preserve one month of state.

Pruning

Operator would still be able to prune runtime and consensus state to any value it wants, however we will not allow pruning of consensus blocks and or runtime storage diffs younger then expired history.

Start by tracing the existing checkpoint creation and replacement flow, including the genesis checkpoint, then inspect how runtime and consensus pruning handles blocks and storage diffs. Done means nodes can serve a periodically replaced history-expiry checkpoint and cannot prune consensus blocks or runtime storage diffs younger than the configured expiry.