nodejs / nodejs/citgm

discussion: try to parse test output for each package (and mark individual test-cases to ignore)

Open
#428 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
606
Forks
157
Avg merge
3d 23h
Merged PRs (30d)
3

Description

I'm sure it's been brought up before, but I'm contemplating ways to parse the test output of each package and enable a finer grained ignore/flaky annotation, optimally on a per test case level, but on a suite level will be good as well.
My idea is to parse the package.json:scripts.test field, deduce test engine and args, and try to call engine directly with a TAP formatter. This could be a "best-effort" approach, with the current behaviour (npm test) as fall back.
AFAICT most packages use a well known test runner (mocha, tap, etc.) that know how to output TAP format.

Feedback is most appreciated...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing each package's package.json:scripts.test field and the existing npm test fallback. Investigate how common test engines can emit TAP, then define how per-test-case or suite-level ignore and flaky annotations would be represented. Done means best-effort parsing works where possible while npm test remains the fallback.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, nodejs
Domain
testing, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.