Feasibility: rewrite polkadot-cli in pure Rust (subxt + polkadot-sdk)

What this issue is about

A feasibility study on rewriting polkadot-cli (currently TypeScript + Bun + polkadot-api v2) in pure Rust on top of subxt + polkadot-sdk crates + paritytech/verifiable.

This is a decision-input ticket, not a "let's start coding" ticket. Read the verdict, the pros/cons, and the risks. The implementation phasing at the bottom is for orientation only — it's deliberately a "nice to have, read last" section. The point is to decide whether to do this, not yet how.

Source: polkadot-cli v1.19.0 — 49 source files (~12K LOC), 47 test files (~15K LOC), comprehensive command surface (query, tx, const, events, errors, apis, extensions, rpc, chain, account, metadata, inspect, hash, sign, parachain, verifiable, completions).

Verdict

Feasibility: 7/10.

Recommendation: dual-track with a hard prove-it gate. Don't commit up front. Do this instead:

1. Phase 0 spike (1.5–2 weeks, one engineer). Build, against nextv2-ah, byte-identical signed extrinsic hex compared to the TS CLI, including at least one custom signed extension via a --ext-style override. Plus byte-identical Bandersnatch member key against existing TS golden vectors. If those bytes don't match in 7 working days, abort or extend the spike — don't wave at it.
2. If the spike passes: commit to Phase 1 (4–6 wks). Ship as dot-rs alongside TS, opt-in. Pull internal Parity user feedback.
3. After Phase 1: decide on the remaining phases based on whether the 60-case parity suite is paying for itself.
4. TS stays at maintenance pace during dual-track. No new features in TS once Rust catches up. Hard-cut name dot-rs → dot once parity is declared. Keep npm package as a binary-downloader shim for 12 months.

The one thing to do before committing anything: the Phase 0 byte-equality artefact for --ext. A spike that ends with "looks the same" is theater. A spike that ends with "here is the extrinsic hex from both CLIs, identical to the last byte" is a real go/no-go signal.

Why 7 and not higher

• 1 point off: the --ext schemaless signed-extension encoder is the single concentrated engineering risk. subxt assumes the SE set is known at compile time; replicating papi's "any JSON, any SE, any chain" requires a dynamic encoder built on scale-value + frame-decode + metadata. ~1–2 weeks of focused work and the kind of mistake that bites a year later.
• 1 point off: subxt is pre-1.0 (0.50.1, 2026-04-27). Quarterly minor releases are still breaking; a 0.5–2 day upgrade tax per quarter for 12–18 months.
• 1 point off: parity-testing is large enough that a careless team would skip it and ship a subtly-wrong CLI.

Why 7 and not lower

• Every required capability has a known Rust dep with a known story (subxt + scale-value + frame-decode + sp-core + schnorrkel + paritytech/verifiable + clap v4 + jsonrpsee + tokio).
• The TS codebase ports almost module-for-module. Tests are subprocess-based and the 442 KB Polkadot v15 metadata fixture binary is portable to a Rust test runner unchanged.
• The wins are concrete and load-bearing (cold start, no Node toolchain on user machines, paritytech/verifiable direct, single binary, embedded smoldot becomes practical).
• The biggest cryptographically-sensitive component shrinks instead of growing: verifiablejs is literally a wasm-bindgen wrapper over paritytech/verifiable (Cargo.toml inspected). Rust calls upstream directly — byte-identical by construction, no WASM init.
• Exit ramps are clear: dual-track preserves options, deprecation is mechanical.

Pros and cons

Strictly better in Rust (concrete, not marketing)

• Cold start. Bun TS CLI: ~120–200 ms before any work begins. Rust release+LTO: ~5–15 ms. For a CLI invoked 50× a day, the single most-noticed improvement.
• No Node/Bun on user machines. Single static binary, ~25 MB stripped (~12 MB UPX). CI runners, internal infra, ARM boxes all get one-line installs.
• paritytech/verifiable direct. Removes the WASM init step (~30–50 ms first call) and the verifiablejs JS wrapper hop. Byte-identical output.
• Memory. query.System.Account --dump is ~80–120 MB resident in TS, ~10–20 MB in Rust.
• Distribution. cargo-dist → GH releases + Homebrew tap + cargo + alpine static. Aligned with subxt-cli, polkadot-omni-node, try-runtime, chopsticks binary norms.
• Concurrent chain ops. tokio + subxt parallelizes cheaper than Promise.all (same code, lower per-op overhead).
• Embedded smoldot. Feature flag in Rust, ~40 MB cost. Practical "no-RPC, no-internet" operation against trusted chain specs.
• No native-module rebuild. Static crypto; no postinstall scripts; no platform-specific subpath imports.

Strictly worse in Rust (concrete)

• Dynamic-API ergonomics. subxt::dynamic is decisively second-class to the codegen path. Every call site is ~3× the LOC and harder to skim. Felt throughout the codebase. unsafeApi.query.System.Account.getValue(addr) → client.storage().fetch(&dynamic("System", "Account", vec![Value::from_bytes(addr)]), block_hash).await?.unwrap().to_value()?.
• Compile times. Clean release build 4–8 min Apple Silicon, 8–15 min x86 CI. Incremental dev: 5–30 s. TS build is ~1 s. Iteration on commands slows noticeably.
• Binary size. ~25 MB stripped, ~65 MB with embedded smoldot. Roughly even-on-disk vs node_modules, worse perception.
• subxt pre-1.0 churn. 0.5–2 days/quarter upgrade tax.
• npm discovery loss. npm search polkadot-cli is a real onboarding surface. Cargo discovery is weaker for non-Rust users. Mitigation: keep npm package as a binary-downloader for 12 months.
• YAML library situation. serde_yaml is officially unmaintained (rustsec advisory-db #2132); serde_yml has provenance concerns; serde_yaml2 works but doesn't preserve key order. Hand-roll --to-yaml output for stability.
• Dynamic --ext porting cost. ~1–2 wks of focused work. The concentrated risk.
• Lost workflow: JS REPL debugging. papi's getUnsafeApi() is excellent as a debug aid alongside the CLI. Rust CLI is results-only. Phase 4's dot-tx crate publication recovers this for Rust users only.
• Drive-by community PRs. The TS CLI's recent history shows non-Parity contributors. Rust raises the contribution bar; this surface will drop ~to zero.

What Rust enables that TS struggles with

1. Embedded light client. smoldot in Rust is a feature flag, not a 5 MB WASM bundle with init cost. Genuine no-RPC operation becomes practical.
2. In-process composition with try-runtime. Rust composes frame-try-runtime directly. TS does subprocess + JSON bridge at best.
3. Reusable dot-meta + dot-tx crates for the ecosystem. There is no good "dynamic papi-equivalent for Rust" today. Indexers, bots, monitoring scripts, signing services all want this. Force multiplier; not achievable from TS.
4. FFI surface. Python/Go/Swift bindings via C ABI become straightforward.
5. WASM target for browser later. cargo build --target wasm32-unknown-unknown of dot-meta/dot-tx is a real product opportunity — dynamic transaction builder for browser dApps that don't ship papi.
6. Static reproducible builds. GitHub-attested binaries, sigstore signing, deterministic builds.
7. Performance ceiling for batch ops. 1M-key storage dump streams cleanly in Rust; V8 GC-pressures the TS path.

What current users lose / break

• npm install -g polkadot-cli path. Phase 4 keeps the package as a binary-downloader for 12 months.
• During dual-track, Rust ships as dot-rs; renames to dot only once parity is declared. TS users keep working through the overlap.
• Hold ~/.polkadot/ schema stable. Don't XDG-migrate as part of this project; that's a separate breaking-change ticket.
• Behavior dependence on papi internals (e.g. the ChargeAssetTxPayment isAssetCompat bypass at tx.ts:340-353). The Rust port won't carry the workarounds. Document explicitly.
• Any commands deferred past Phase 3: smoldot light mode (optional), update notifier, niche outputs like --raw SCALE hex on edge categories. Track as known gaps, not silent drops.

Capability matrix (the technical grounding for the verdict)

Verified against current crates.io / docs.rs / GitHub state (2025-2026).

Capability #2 is the only one whose risk is concentrated. Everything else is mechanical or well-served.

Capability #6 is the upside that flipped this from "be cautious" to "feasible". The cryptographically-sensitive component shrinks instead of growing.

Risks that move the estimate

In priority order:

1. Dynamic --ext byte-equality mismatch. If scale_value::scale::encode_as_type produces different bytes from papi's builder.buildDefinition(typeId).enc(value) for any non-trivial SE, debugging is multi-day SCALE-spec archaeology per case. Mitigation: Phase 0 hard gate against nextv2-ah.
2. subxt 0.x minor churn during Phases 1–2. Pin Cargo.lock. Budget 1 day per upgrade.
3. Schnorrkel HDKD edge cases. TS mirrors @polkadot-labs/hdkd-helpers internals (accounts.ts:56-89) because they aren't re-exported. sp-core must produce identical chain codes for numeric and string junctions. Mitigation: Phase 0 golden-vector test.
4. YAML round-trip. serde_yaml2 doesn't preserve key order; TS yaml package does. Mitigation: hand-serialize fixed-order YAML for --to-yaml.
5. --ext JSON normalization. The shape papi accepts is loose (bare value vs {value}, enums as {type, value} or string, BigInt as string or number). Port parseTypedArg + normalizeValue (tx.ts:1100-1540) faithfully.
6. Output formatter surface. ~30 small format functions in tx.ts. Tedious not hard. ~3–5 days, not "trivial".

Parity testing strategy (the single most important non-code investment)

The failure mode this defends against: "ship a Rust CLI with 95% parity, where 5% silently produces wrong bytes."

Two-CLI diff harness, co-located in this repo

// tests/parity/run_both.rs
struct ParityCase { name, args, expects: ByteIdentical | JsonEqual | SemanticEqual(fn) }
fn run_ts(args, env) -> Output { /* exec `bun src/cli.ts` */ }
fn run_rs(args, env) -> Output { /* exec `target/release/dot` */ }

ByteIdentical for crypto-driven goldens (addresses, encoded calls, signatures, storage keys). JsonEqual for --json (key order may differ; use indexmap if it must be preserved). SemanticEqual(fn) for content with timestamps or other non-determinism.

Goldens are TS-CLI-captured JSON outputs, regenerated by a script. Rust diffs against them. Intentional TS behavior change → regen + review in a PR. Unintentional drift → caught at CI time.

Test-category strategy

The 60-case must-pass parity suite (gating release)

All 60 must pass byte-identically (modulo platform newlines + ANSI under --json).

• Crypto (15): SS58 prefix 0/2/42, sign hello/0xhex, mnemonic+derivation pubkey, pallet+parachain sovereigns (treasury / 1000-child / 1000-sibling), bandersnatch member, blake2-128/256, xxh64, keccak256.
• SCALE encoding (15): Balances.transfer_keep_alive, System.remark, nested Utility.batch, XCM Location PolkadotXcm.send, asset transfer on asset-hub, compact BigInt, raw tx.0x1f... decode, --to-yaml/--to-json round-trips, storage-key hash + decoded value, runtime API apis.Core.version.
• Dynamic --ext (10): {} produces same hex as no-ext, CheckMetadataHash override, unsigned/general v5 0x45, --asset XCM Location, full custom SE on paseo-asset-hub, value-and-additionalSigned override, shape-mismatch error category equivalence, era=immortal exact bytes, era=Mortal64 exact bytes, tip+nonce+mortality combined.
• Submission flow on chopsticks (10): dry-run fee ±1%, broadcast event txHash, best-block events, finalized dispatch result, unsigned Sudo bypass, --at against pinned block, Balances.InsufficientBalance decode, Balances.Transfer event decode, NDJSON byte-equal, mortality expiry behavior.
• CLI surface (10): --help, completions script validity, chain list --json, chain add + read-back on-disk, file-based ./transfer.yaml --to-json, --var substitution, --version, fuzzy-match suggestions text, --at 0xshort error, __complete dot quer<TAB>.

CI layout

• Pre-merge gate: 60 cases against the fixture binary only (zero network). ~90s added to CI.
• Nightly: chopsticks-backed for the 10 submission-flow cases. 5–10 min.
• Quarterly manual: Westend/Paseo sanity run before each release.

Verification spike (do this before acting on this issue)

Three smoke tests before committing to the Phase 0 spike. If any fails, the assumptions here need revision.

1. paritytech/verifiable builds in isolation:

   cargo new --bin verifiable-spike && cd verifiable-spike
   cargo add verifiable --git https://github.com/paritytech/verifiable.git
   cargo build
   

   ≤ 5 min, no toolchain errors.
2. subxt::dynamic against nextv2-ah: 20-LOC storage query from current docs.rs/subxt.
3. Metadata fixture loads from Rust: 5-LOC integration test that loads src/commands/__fixtures__/polkadot-metadata.bin via frame-metadata + scale-info and prints pallet names.

dot/
├── crates/
│   ├── dot-meta/      # MetadataBundle, lookup, fingerprint, describe, fetch
│   ├── dot-tx/        # Dynamic call codec + SE encoder + general-tx (the risky bit)
│   ├── dot-keys/      # sr25519 + BIP39 + SURI + SS58 + bandersnatch + sovereigns
│   ├── dot-client/    # ChainClient over Legacy + Unstable backends; submit-and-watch
│   └── dot-cli/       # [[bin]] name = "dot": clap, commands, config I/O, output
└── tests/
    ├── fixtures/polkadot-metadata.bin   # symlink → src/commands/__fixtures__/polkadot-metadata.bin
    └── parity/                           # cross-CLI diff harness

Critical files to read before starting Phase 0

• src/commands/tx.ts — the largest, riskiest command; SE encoder lives here.
• src/core/metadata.ts — metadata fetch + decode + fingerprint + persistence.
• src/core/client.ts — client lifecycle, lazy connect, destroy semantics.
• src/core/accounts.ts — sr25519 + BIP39 + SURI + sovereign derivation.
• src/commands/__fixtures__/run-cli.ts — runCli pattern that the Rust runDot must mirror 1:1.
• src/utils/runtime-fingerprint.ts — fingerprint shape; Rust cache must match for dual-track compat.
• src/commands/__fixtures__/polkadot-metadata.bin — symlink target for Rust test fixture.

Start with the Verdict and Phase 0 prove-it gate, then read the referenced TypeScript paths tx.ts:1100-1540 and accounts.ts:56-89 plus the proposed tests/parity/run_both.rs harness. This issue is complete when the feasibility decision is supported by the specified byte-equality and golden-vector evidence; it is not yet a self-contained coding task.