pytest-dev / pytest-dev/pytest

stdout capturing breaks sys.stdout.encoding

Open
#4,389 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

plugin: capture type: bug
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

py.test by default replaces sys.stdout with something that captures test output, which is great! But the replacement stdout doesn't have a correct encoding attribute, and that can break the code under test.

(Prior to Python 3.6 sys.stdout.encoding is the only way to figure out the current OEM encoding on Windows, which you need if you have to decode the output of subprocesses. You can see what happens when people try to use py.test on such code here: https://github.com/mgedmin/check-manifest/pull/92)

Here's a reproducible test case for Linux (note that you need to use Python 3.5 or older):

# test.py
import sys

def test_stdout_encoding():
    assert sys.stdout.encoding == 'ANSI_X3.4-1968'
LC_ALL=C py.test test.py -s     # succeeds
LC_ALL=C py.test test.py        # fails, sys.stdout.encoding is 'UTF8'
$ pip list|grep pytest
pytest                        3.9.3        
pytest-catchlog               1.2.2        /home/mg/src/pytest-catchlog
pytest-forked                 0.2          
pytest-xdist                  1.24.0       

This is similar to #2375, except I'm not using the capsys fixture.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the test.py reproduction under Python 3.5 or older, comparing LC_ALL=C with and without -s. Trace pytest's default stdout capture and verify that the captured stream preserves the encoding expected by code under test; done means the reproduction passes without -s.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.