[Provider/Runtime] Version guard: protocol-version handshake between provider and chain

Subtask of #254: a protocol-version handshake so a provider detects incompatibility with the runtime instead of mis-decoding or failing mid-operation.

Motivation

There is currently nothing that tells a running provider "you are too old / too new for this runtime" — the provider connects, caches metadata once, and keeps going. Today, while the provider still hand-parses storage values and keys at fixed byte offsets (provider-node/src/challenge_responder.rs, subxt_client.rs), a runtime upgrade that shifts a layout doesn't produce a clean error — it produces silently wrong values. The worst case is concrete: the challenge responder decodes garbage or skips entries, challenges go unanswered for 48h, and the provider is slashed its entire stake (slash_provider_for_failed_challenge).

The migration away from subxt::dynamic + manual decoding toward the typed, versioned StorageProviderApi (#243, follow-ups to #237) is already on the way, and it removes the silent mis-decode class. But it does not remove the need for a guard — it changes the question the guard must answer:

When a runtime upgrade bumps StorageProviderApi to v2 and the v1 methods the provider speaks disappear (or change semantics), how does a running provider react?

Without a guard, the answer is: the next runtime-API call fails at call time (or worse, a renamed-but-compatible-looking method returns something the provider misinterprets), in whatever coordinator happens to hit it first — checkpoint loop, challenge responder — with no coherent overall reaction. This failure class has in fact already occurred once: CommitmentPayload::CURRENT_VERSION was bumped 1→2 when the nonce field was added — a provider still signing v1 payloads after that change gets every commit/checkpoint signature rejected on-chain, with nothing telling it why. A version guard turns both failure classes ("silently mis-decode" now, "API version vanished underneath me" after the migration) into one defined behaviour: refuse loudly / degrade safely, alert the operator. It is the cheapest piece of #254, a prerequisite for any self-update path, and it gives #214 its first meaningful alert.

Current state

• SubxtChainClient::connect = OnlineClient::from_url + signer setup, no compatibility assertion of any kind (provider-node/src/subxt_client.rs). No spec_version / transaction_version / metadata / runtime-API-version check anywhere in provider-node or client.
• No reaction to runtime upgrades. The chain-state coordinator subscribes to finalized blocks but filters events to the StorageProvider pallet only (chain_state_coordinator.rs) — a runtime upgrade is invisible. subxt's cached metadata is refreshed only incidentally on reconnect.
• The auth.rs chain client is cached in a OnceCell and deliberately never reconnects — after an upgrade it keeps stale metadata indefinitely.
• On-chain there is no provider-facing version signal: no protocol-version constant or storage item, and StorageProviderApi (pallet/src/runtime_api.rs) carries no #[api_version] (defaults to 1), so the API surface the migration is standardizing on has no versioning discipline yet either.
• The design does already version the provider-facing signed payloads: CommitmentPayload / CheckpointProposal carry a version: u8 precisely "so the protocol can evolve without breaking existing signatures" (see docs/design/scalable-web3-storage-implementation.md), and CommitmentPayload::CURRENT_VERSION is already at 2. But enforcement is implicit (wrong version ⇒ signature verification fails) and there is no way for a provider to discover the expected version — the missing explicit layer this issue adds.
• The only existing precedent for "provider reads chain config at connect" is the RequestTimeout pallet constant fetch (sync_constants in chain_state_coordinator.rs) — the right pattern to extend.

Proposed / potential solution 1

• On-chain: a provider-facing ProtocolVersion #[pallet::constant] (major/minor; distinct from spec_version, bumped only when providers are affected), plus #[api_version(N)] on StorageProviderApi. Breaking changes keep the old api_version implemented for a deprecation window (sp-api supports implementing multiple versions), so an API version never disappears in the same upgrade that introduces its replacement. ProtocolVersion acts as the queryable umbrella over the existing signed-payload versions — bumping a payload CURRENT_VERSION is by definition a ProtocolVersion major bump.
• Provider: declares the ProtocolVersion range and api_version(s) it supports, and checks them at startup, on reconnect, and on observing a runtime upgrade (runtime-version subscription + metadata refresh — also fixes the stale-metadata client in auth.rs).
• On mismatch, never mis-decode and never fail mid-coordinator: enter a sticky, visible degraded read-only mode (keep serving HTTP reads, stop chain writes and chain-state decoding), report via /health (and /ready, #258), alert. Minor drift / open deprecation window → log + metric only.
• Process rule: any runtime PR that changes the provider-facing surface must bump ProtocolVersion in the same PR (CI-checkable via metadata diff).

Proposed / potential solution 2

• TODO: investigate more the connection and dependency between provider and chain

Related

• #254 — parent (the versioned RuntimeApi and storage-decoupling work build on this guard)
• #243 / #174 — the dynamic→typed/runtime-API migration this guard is designed around
• #258 — graceful shutdown / /ready; degraded mode and exit-with-intent share that surface
• #222 — reconnect/light-client work shares the runtime-version subscription
• #214 — compatibility metric + alert
• #230 — same "config can drift at runtime" theme (try_state for on-chain invariants)

Start with SubxtChainClient::connect, auth.rs, chain_state_coordinator.rs, and pallet/src/runtime_api.rs; compare their connection, metadata, runtime-upgrade, and API-version responsibilities with the signed-payload versioning in docs/design/scalable-web3-storage-implementation.md. Done means the compatibility contract, upgrade detection, mismatch behavior, and provider-facing version signal are specified clearly enough to guide implementation.