[Provider] Storage data-format versioning & migration (fail loudly on incompatible on-disk data)

Motivation

The provider's persisted state (RocksDB today) has no format identity: nothing on disk says which encoding or schema version wrote it. Any change to the serialization silently invalidates existing provider data, and the node does not notice at startup — it limps along treating undecodable buckets as missing.

PR #365 is the first real instance: switching bincode → parity-scale-codec means data written before the upgrade no longer decodes. The failure mode today is soft and quiet: get_bucket logs a tracing::warn! and returns None, which callers surface as Error::BucketNotFound — an operator sees "bucket not found" errors instead of "your database format is incompatible, resync/migrate required".

SCALE makes this class of problem sharper: the encoding is purely positional (no field names or tags on disk), so even reordering fields of BucketState forks the disk format with no compiler or test signal — and same-typed field swaps decode "successfully" with values scrambled.

Whatever DB engine we end up with (#101), we need a story for evolving the on-disk format without stranding or corrupting existing provider data.

Current state

• crates/providers/storage/src/backend/disk.rs — BucketState and StoredNode are SCALE-encoded raw into column families; decode failures warn and return None.
• A metadata column family already exists (CF_METADATA, currently holding only the persisted nonce counter under KEY_NONCE) — a natural home for a format-version marker.
• DiskStorage::new opens the DB unconditionally; there is no compatibility check of any kind.
• No golden-vector tests pin the encoded byte layout of persisted types (round-trip tests added in #365 pass even after a format-breaking field reorder).

Proposed / potential solutions

• Format-version marker in CF_METADATA, written on DB creation and checked in DiskStorage::new: unknown/missing version ⇒ refuse to start with an actionable error (resync or run migration) instead of per-value decode warnings.
• Migration hooks keyed off that marker: on version mismatch, run registered migrations (or, for a storage provider, possibly the cheaper answer: wipe and resync from chain/replicas — worth deciding explicitly).
• Golden-vector tests for every persisted type (assert_eq!(hex::encode(value.encode()), "…")) so an accidental format change fails CI.
• Keep the mechanism behind StorageBackend / engine-agnostic, so the engine evaluation in #101 can swap the DB without changing the versioning contract.

TODO

• Decide the migration philosophy for provider data: migrate in place vs. wipe-and-resync per version bump
• Add format-version marker to CF_METADATA + startup check in DiskStorage::new
• Add golden-vector encoding tests for BucketState and StoredNode
• Document the version-bump procedure for future format changes (when to bump, how to add a migration)

Related issues

• #101 — DB engine evaluation (engine choice constrains the migration mechanics)
• #36 — default storage/bucket internal implementation
• #365 — bincode → SCALE (the breaking change that motivated this)
• #254 / #259 — provider↔chain protocol versioning; distinct concern, but the startup guard could live next to that handshake

Start with crates/providers/storage/src/backend/disk.rs, especially BucketState, StoredNode, CF_METADATA, and DiskStorage::new. Review the related issues #101 and #365 before deciding whether incompatible data should be migrated or wiped and resynced. Done means the chosen policy is implemented with startup compatibility handling, golden-vector tests, and documented version-bump procedures.