paritytech / paritytech/polkadot-cli

Centralize stdout output through a drain-safe helper to prevent pipe truncation

Open
#198 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
10
Forks
2
Avg merge
12h 35m
Merged PRs (30d)
4

Description

Context

PR #194 fixed a Linux pipe-truncation bug in dot metadata <chain>: console.log writes are async, and process.exit() doesn't drain Bun's userspace stdout buffer, so multi-MB output gets cut off when piped (e.g. into jq). The fix uses process.stdout.write(text, callback) and awaits the callback before the command returns.

That fix is targeted to src/commands/metadata.ts because metadata is the only command that reliably produces multi-MB output today. But the same latent bug applies to any command whose output crosses the pipe-buffer threshold (~64KB on Linux).

Likely future offenders

  • dot query --dump — dumps all entries of a storage map; can easily reach MBs on a busy pallet.
  • dot inspect <Pallet> — large pallets with full call/event/error/storage listings.
  • Any future JSON or table output that doesn't bound its size.

These will silently truncate on Linux pipes. Local macOS happens to win the race most of the time, so the regression is easy to miss in dev and only shows up in CI or production Linux pipes.

Proposal

  1. Move the writeStdout helper from src/commands/metadata.ts into src/core/output.ts (or a new src/core/stdio.ts) as the single, drain-safe primitive: process.stdout.write(text, cb) wrapped in a Promise, EPIPE swallowed.
  2. Route printResult, printJsonLine, printHeading, printItem, etc. through it (or expose writeOutputAsync for callers that want to await).
  3. Update call sites in commands that emit potentially large output (metadata, query --dump, inspect) to await the helper before returning.
  4. Drop the metadata.ts-local helper.

Tests can keep mocking via process.stdout.write-patching (the pattern in src/commands/metadata.test.ts:186 after #194) — small refactor to share that helper across test files would be nice.

Out of scope

  • Switching to a logging library (pino/consola) — overkill; we're emitting structured data, not logs.
  • Restructuring stderr/diagnostic output — that's a separate concern.

Verification

A repro for the bug class: any command that outputs >64KB through a pipe on Linux. Until centralized, run each candidate against jq or wc -c and compare to file-redirected size.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the helper in src/commands/metadata.ts and the output functions named in the issue, then inspect src/commands/metadata.test.ts around line 186 and the metadata, query --dump, and inspect call sites. Verify large piped output against file-redirected output using jq or wc -c, and run the affected tests after the shared helper is used.

Written by the indexing model from the issue text.

Assessment

Tech stack
bun, linux, typescript
Domain
cli, operating-systems
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.