cdm deploy races registration against the Bulletin publish — a publish failure leaves the package registered but permanently uninstallable

Summary

A Bulletin metadata publish failure during cdm deploy leaves the package registered but permanently uninstallable, and retrying makes it worse: the retry deploys a fresh contract set at the next version, orphaning the first. Hit twice, identically, deploying three contracts to -n paseo with cdm 0.13.0.

[Bulletin publish item 1/3] Metadata publish failed (metadata-0):
  [Bulletin publish] Transaction timed out after 300s.
  The transaction may still be processing on-chain.

After the failure:

• the contracts are instantiated and are registered — getVersionCount incremented, getAddress/getMetadataUri return the new address and CID;
• cdm deploy exits non-zero and cdm.json is never written;
• cdm install resolves address + CID from the registry, then dies on IPFS fetch failed: Gateway Timeout — the gateway 504s exactly the new CIDs (an old CID: HTTP 200 in 0.49s; the new ones: 504, polled every 20s for 4 minutes). The bytes never reached the Bulletin chain, so nothing can serve them.

Root cause (cdm side)

src/lib/contracts/src/pipeline.ts submits the two chain writes concurrently:

const [deployRes, publishRes] = await Promise.all([
    deployer.deployAndRegisterBatch(..., metadataUris, ...), // Asset Hub: instantiate + registry publishLatest(CID)
    publisher.publishBatch(metadataList),                    // Bulletin: TransactionStorage.store
]);

The CID is precomputed locally and the Asset Hub batch registers it atomically with the instantiate — so by the time the Bulletin store times out, the registry already points at content that doesn't exist. The Promise.all rejection can only abort the pipeline after the damage is done.

Recovery is then impossible by design:

• the metadata embeds published_at: new Date().toISOString(), so a re-run produces different bytes → a different CID — the bytes hashing to the registered CID only ever existed in memory and are gone;
• there is no publish the metadata I already registered path;
• a retried deploy reads the bumped getVersionCount, derives a new CREATE2 salt, and instantiates a second contract set — we now have an orphaned v0 and a broken v1 on paseo.

(Why Bulletin never included a validly broadcast TransactionStorage.store within 300s — twice — is a separate chain-side question; allowance was healthy: 100 txns, 40.9 MB. This issue is about cdm turning that transient failure into a permanent one.)

Fix

Sequence the writes: publish to Bulletin, verify CIDs, then deploy+register. This restores the invariant registered ⇒ fetchable. A failed publish aborts before anything is registered, so the deploy is safely retryable; a deploy failure after a successful publish leaves only harmless orphaned Bulletin content.

Implemented and validated against a real deploy; PR to follow.

Follow-ups worth considering (not in the PR)

1. Persist the metadata bytes under .cdm/ (keyed by CID + package/version) before submitting, so an interrupted run leaves a re-submittable artifact.
2. cdm publish-metadata <package> — re-submit persisted bytes whose CID matches the registry's getMetadataUri, as a repair path for states like the current paseo one.
3. Surface the tx hash in TxTimeoutError reporting (product-sdk-tx logs it, the error doesn't carry it), so "may still be processing" is checkable.

Environment

• cdm 0.13.0, bun 1.2.23, macOS arm64
• chain: paseo preset; registry 0xc1a73a4f93fde65b1cb1680baead248073566cb0
• gateway: https://paseo-bulletin-next-ipfs.polkadot.io/ipfs/
• affected packages: @w3s/playground-registry, @w3s/playground-identity, @w3s/playground-open-verifier (v0 orphaned + v1 registered-but-unfetchable)

Start in src/lib/contracts/src/pipeline.ts and trace the cdm deploy flow around the concurrent deployAndRegisterBatch and publishBatch calls. Verify the failure and retry behavior, then validate a deploy against the affected chain. Done means Bulletin content is available before registry registration, and a publish failure leaves deployment safely retryable.