janestreet / janestreet/ppx_inline_test

Don't show test output unless tests fail

Open
#16 4 comments 0 reactions 0 assignees View on GitHub
forwarded-to-js-devs
Dominant language
OCaml
Stars
143
Forks
25
PR merge metrics
No merged PRs in 30d

Description

Our code does a lot of logging, which makes it really hard to quickly tell which test failed when looking at our test output. Is there a way ppx_inline_test could hide stdout unless a test fails?

I looked into implementing this myself with `Runtime.am_running_inline_test`, but I would also need a way to know when a test starts and stops, and also know whether it passed or failed.

(One option is to get rid of all output when tests are running, but then it's really difficult to debug failing tests)

I could also implement my own test runner, but it's not clear to me what I would need to do there since the built-in runner just calls `Ppx_inline_test_lib.Runtime.exit ()`.

Contributor guide

Open the contributing guide

Research direction

Start with Ppx_inline_test_lib.Runtime.am_running_inline_test and the built-in runner entry point, Runtime.exit, to understand how tests begin, end, and report results. Define how stdout is retained for failing tests while being suppressed for passing tests, then verify the runner still exposes useful output when a test fails.

Written by the indexing model from the issue text.

Assessment

Tech stack
ocaml
Domain
testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.