maehr / maehr/quarto-cli-mcp

test: one test fails at random in the pre-push gate

Open
#32 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
1
Forks
0
Avg merge
23m
Merged PRs (30d)
17

Description

Problem

pnpm run check failed three times in the pre-push hook during one session. Each failure reported
exactly one failed test. A rerun passed every time. CI never failed.

Observations

When Result
Push of fix/stdio-shutdown the hook failed, the retry passed
Local run after the #23 work `1 failed
Push of docs/zenodo-doi the hook failed, the retry passed

The name of the failed test was not captured. The second failure happened after
test/tools/metadata-merge.test.ts existed, the first one before it, so the flake is not limited to
one file.

Not reproducible on demand

The full suite passed 12 times in a row afterwards, both with and without coverage.
test/stdio.test.ts passed 8 times in a row on its own.

A likely cause

Vitest runs the files in parallel. Several tests start the real Quarto CLI, and one starts a child
Node process. A loaded machine makes those slower. The pre-push hook runs while the developer does
other work, which is the one condition that the clean runs did not have.

Next step

Capture the name first. Add a reporter that keeps the failure, for example
vitest run --reporter=junit --outputFile=..., or run the hook with --retry=0 --bail=1 and keep
the log. Fix nothing until the failing test is known.

Not a blocker

The v0.2.0 release passed the gate on Node 22 and Node 24.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the pre-push hook and pnpm run check; rerun with --retry=0 --bail=1 while retaining the output, or use the suggested JUnit reporter. Review test/tools/metadata-merge.test.ts and test/stdio.test.ts as the named comparison points. Done means the randomly failing test is identified and the flake is addressed, with the check passing reliably.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, typescript
Domain
developer-experience, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.