pytest-dev / pytest-dev/pytest

`capfd` appears to be flaky on Windows (sometimes returning an empty string)

Open
#10,843 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

platform: windows plugin: capture type: bug
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

The nanobind test suite occasionally runs into a flake (*) where capfd does not correctly capture output printed to stderr. The structure of the test looks as follows:

def test_01(capfd):
    c_function()
    captured = capfd.readouterr()
    assert captured.err.strip() == "<some string>"

Here, c_function() is a function implemented outside of Python (bound via the C API) that invokes

fprintf(stderr, "<some string>\n");

This works reliably on Linux and on macOS. On Windows, every once in a while, the test fails, and the contents of captured.err.strip() are empty.

In the linked error report, testing took place on GitHub actions with Python 3.10 and PyTest 7.2.2. When the flake occurs, it is usually enough to re-run the test and it passes.

I am quite sure that the capfd or I/O system are somehow to blame -- we previously used Python warnings to capture the output, which was rock-solid for many months. It was only once I started switching to the capfd/fprintf combination that the flaky behavior started to emerge.

I would be grateful if you have ideas.

Many thanks!
Wenzel

(*) for example, this one: https://github.com/wjakob/nanobind/actions/runs/4533051238

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the capfd usage described in the issue and the nanobind GitHub Actions run linked there, comparing the Windows failure with the Linux and macOS behavior. Reproduce the intermittent empty stderr capture around c_function() and fprintf(stderr, ...); done means the relevant capture behavior is reliable or the failure is narrowed to a documented cause.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
operating-systems, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.