paritytech / paritytech/host-api-test-sdk
Fault-injection mock host: latency, dropped handshake, flaky transport
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 2
- Forks
- 1
- Avg merge
- 3h 46m
- Merged PRs (30d)
- 2
Description
Summary
Extend the strict mock host with fault-injection knobs so SDK-level failure modes are reproducible in CI: injected latency, a dropped/never-answered handshake (a permanent repro for product-sdk#200), a flaky transport, and a version-skewed host. Today these are only reproducible against real hosts, by hand, after the fact.
Motivation
The test host is faithful (real signing, real SCALE, SS58-only matching) but always happy-path. The bugs that bite are timing/transport artifacts a canned mock can't reproduce - product-sdk#200 (a 0.8.4 client silently drops the handshake against a 0.8.0 host; isReady() hangs), and the desktop ChainHead churn hang. A fault layer over the real transport reproduces them deterministically.
Design
A thin layer wrapping the raw @novasamatech/host-api Provider, read on every message so faults work up front and at runtime:
createTestHostServer({
productUrl,
faults: { latencyMs?, dropHandshake?, dropEveryNth?, protocolVersion? },
});
Plus FAULT_SCENARIOS presets (droppedHandshake, flakyTransport, versionSkew, slowSigning, highLatency) and runtime setFaults/getFaults on the fixture. Handshake frames are recognised by reading the SCALE-compact requestId length + the MessagePayload enum index (response = 1, request = 0) - no dependency on non-exported codecs. protocolVersion rewrites only the inbound handshake request's trailing codec-id byte, so the host's built-in handler emits the real UnsupportedProtocolVersion (no fake/encoding on our side). No-fault config is a transparent pass-through.
Done (feature-complete, green locally)
- Fault layer (
src/fault-provider.ts), DOM-free; bundled by esbuild + unit-testable under node. -
faultsoption +setFaults/getFaultsthrough runtime + Playwright fixture;FAULT_SCENARIOS. -
dropHandshake(product-sdk#200 repro),latencyMs,dropEveryNth,protocolVersion(version skew - realUnsupportedProtocolVersionpath). - 13 unit cases + 5 E2E specs; full integration suite 61/61 green.
- Mandated docs updated (CHANGELOG, forum-post, README).
Out of scope - deferred to the @parity/truapi-host engine swap
-
flakyTransportrecovery (true retry): asserting the SDK recovers a dropped request needs a per-request timeout/retry in the transport and a retry-capable signer in the loop - neither exists on the@novasamatechengine (a dropped request hangs with nothing to reject on). The shippedflakyTransportproves the fault is observable (the call stalls); recovery follows[ENG]. - Literal-semver version negotiation: the wire carries a u8 codec id, not a semver. A semver-string negotiation needs the
@parity/truapi-hostengine.
Acceptance criteria
faults+setFaults/getFaultsdocumented and exported.droppedHandshakeE2E fails the way product-sdk#200 fails.versionSkewE2E: handshake rejected, product does not connect.flakyTransportE2E: a dropped signing request is observably stalled.- No-fault config is a transparent pass-through (existing tests unaffected).
Notes
This is migration-independent - it ships on the current @novasamatech engine and does not wait for product-sdk#186. Implementation is complete and green locally; the PR is a rebase onto current main + a version re-pick (0.10.0 already shipped without this → 0.11.0) away.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/fault-provider.ts, then inspect the runtime and Playwright fixture exports for faults, setFaults, getFaults, and FAULT_SCENARIOS. Run the 13 unit cases and 5 E2E specs, followed by the full integration suite, and verify the documented acceptance criteria and no-fault pass-through on current main.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- playwright, typescript
- Domain
- testing-qa
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 15/100