Disk-backed externalities

Original issue https://github.com/paritytech/substrate/issues/14076

Prelude

Discussion in https://github.com/paritytech/substrate/issues/13562 expanded to include various topics other than lazy-download, including improving general performance and using a disk-backed database.

I am creating this issue to separate the discussion and progress around allowing the try-runtime-cli to accept a DB path for reading state, rather than loading all state into memory.

Motivation

Creating snapshots and loading state into memory is not a significant issue for chains like Polkadot and Kusama, but it can be for heavier-weight chains like Moonbeam with extensive state.

On these heavy chains:

1. It takes a long time to create a snapshot, and the snapshot size is considerable.
2. The state might not fit into memory, depending on the chain and machine specifications.
3. It would provide a much better developer experience for using try-runtime-cli with these chains if developers could skip worrying about that and just specify the path to their existing database instead.

Implementation ideas

• TestExternalities currently has a hardcoded InMemoryBackend as a generic parameter. However, that could be updated to a more generic sp_state_machine::Backend.
• Once that change is made, we should be able to use either an InMemoryBackend or DiskDbBackend (which is already implemented in fudge) in TestExternalities.
• We can add a new Mode to remote-externalities called something like Disk, so we would have Online, Offline (perhaps rename this to Snap), and Disk.
• Finally, we can update the CLI to accept live, snap, and a new option db when specifying where to get state. If the user chooses db, they pass the path, and the TestExternalities will be initialized with a DiskDbBackend instead of the InMemoryBackend.

Beginne mit TestExternalities, seinem fest codierten InMemoryBackend-Parameter und der im Issue beschriebenen Handhabung von remote-externalities Mode. Überprüfe das Interface sp_state_machine::Backend und die bestehende DiskDbBackend-Implementierung in fudge und verfolge anschließend, wie die CLI derzeit Live- und Snap-State-Quellen auswählt. Die Aufgabe ist erledigt, wenn ein Datenbankpfad State auswählen kann, der von einer Festplatte gelesen wird, ohne den vollständigen Snapshot in den Speicher zu laden.