paritytech / paritytech/host-api-test-sdk

Fault-injection mock host: latency, dropped handshake, flaky transport

Open
#41 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
2
Forks
1
Avg merge
3h 46m
Merged PRs (30d)
2

Description

Summary

Extend the strict mock host with fault-injection knobs so SDK-level failure modes are reproducible in CI: injected latency, a dropped/never-answered handshake (a permanent repro for product-sdk#200), a flaky transport, and a version-skewed host. Today these are only reproducible against real hosts, by hand, after the fact.

Motivation

The test host is faithful (real signing, real SCALE, SS58-only matching) but always happy-path. The bugs that bite are timing/transport artifacts a canned mock can't reproduce - product-sdk#200 (a 0.8.4 client silently drops the handshake against a 0.8.0 host; isReady() hangs), and the desktop ChainHead churn hang. A fault layer over the real transport reproduces them deterministically.

Design

A thin layer wrapping the raw @novasamatech/host-api Provider, read on every message so faults work up front and at runtime:

createTestHostServer({
  productUrl,
  faults: { latencyMs?, dropHandshake?, dropEveryNth?, protocolVersion? },
});

Plus FAULT_SCENARIOS presets (droppedHandshake, flakyTransport, versionSkew, slowSigning, highLatency) and runtime setFaults/getFaults on the fixture. Handshake frames are recognised by reading the SCALE-compact requestId length + the MessagePayload enum index (response = 1, request = 0) - no dependency on non-exported codecs. protocolVersion rewrites only the inbound handshake request's trailing codec-id byte, so the host's built-in handler emits the real UnsupportedProtocolVersion (no fake/encoding on our side). No-fault config is a transparent pass-through.

Done (feature-complete, green locally)

  • Fault layer (src/fault-provider.ts), DOM-free; bundled by esbuild + unit-testable under node.
  • faults option + setFaults/getFaults through runtime + Playwright fixture; FAULT_SCENARIOS.
  • dropHandshake (product-sdk#200 repro), latencyMs, dropEveryNth, protocolVersion (version skew - real UnsupportedProtocolVersion path).
  • 13 unit cases + 5 E2E specs; full integration suite 61/61 green.
  • Mandated docs updated (CHANGELOG, forum-post, README).

Out of scope - deferred to the @parity/truapi-host engine swap

  • flakyTransport recovery (true retry): asserting the SDK recovers a dropped request needs a per-request timeout/retry in the transport and a retry-capable signer in the loop - neither exists on the @novasamatech engine (a dropped request hangs with nothing to reject on). The shipped flakyTransport proves the fault is observable (the call stalls); recovery follows [ENG].
  • Literal-semver version negotiation: the wire carries a u8 codec id, not a semver. A semver-string negotiation needs the @parity/truapi-host engine.

Acceptance criteria

  • faults + setFaults/getFaults documented and exported.
  • droppedHandshake E2E fails the way product-sdk#200 fails.
  • versionSkew E2E: handshake rejected, product does not connect.
  • flakyTransport E2E: a dropped signing request is observably stalled.
  • No-fault config is a transparent pass-through (existing tests unaffected).

Notes

This is migration-independent - it ships on the current @novasamatech engine and does not wait for product-sdk#186. Implementation is complete and green locally; the PR is a rebase onto current main + a version re-pick (0.10.0 already shipped without this → 0.11.0) away.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with src/fault-provider.ts, then inspect the runtime and Playwright fixture exports for faults, setFaults, getFaults, and FAULT_SCENARIOS. Run the 13 unit cases and 5 E2E specs, followed by the full integration suite, and verify the documented acceptance criteria and no-fault pass-through on current main.

Written by the indexing model from the issue text.

Assessment

Tech stack
playwright, typescript
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.