report_internal_error() alternates stdout and stderr without flushing properly

Open
#4,458 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
2/5
Estimated time
1-3 hours
Newbie friendliness
52/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
python
Domain
tooling

Research direction

Start by locating report_internal_error() in the mypy source and read how it writes traceback and error messages to stdout and stderr. Reproduce or inspect the internal-error reporting path, then verify that the traceback and diagnostic messages are flushed in the intended order.

Written by the indexing model from the issue text.

Description

A recent internal mypy run ended in a crash, and look at the output:

  File "/srv/server/.mypy/venv/lib/python3.5/site-packages/mypy/typeanal.py", line 727, in visit_instance
    if not is_subtype(arg, bound):
  File "/srv/server/.mypy/venv/lib/python3.5/site-packages/mypy/subtypes.py", line 83, in is_subtype
    ignore_declared_variance=ignore_declared_variance))
  File "/srv/server/.mypy/venv/lib/python3.5/site-packages/mypy/types.py", line 1230, in accept
    return visitor.visit_union_type(self)
  File "/srv/server/.mypy/venv/lib/python3.5/site-packages/mypy/subtypes.py", line 312, in visit_union_type
metaserver/lib/cloud_docs/event_ordering.py:1: error: INTERNAL ERROR -- please report a bug at https://github.com/python/mypy/issues version: 0.560-dev-fbd23bfd20df2920f878e9e55e39f4aa71919319
metaserver/lib/cloud_docs/event_ordering.py:1: : note: use --pdb to drop into pdb
    for item in left.items)
  File "/srv/server/.mypy/venv/lib/python3.5/site-packages/mypy/subtypes.py", line 312, in <genexpr>
    for item in left.items)
  File "/srv/server/.mypy/venv/lib/python3.5/site-packages/mypy/subtypes.py", line 83, in is_subtype
    ignore_declared_variance=ignore_declared_variance))
  File "/srv/server/.mypy/venv/lib/python3.5/site-packages/mypy/types.py", line 1415, in accept
    return visitor.visit_forwardref_type(self)
  File "/srv/server/.mypy/venv/lib/python3.5/site-packages/mypy/types.py", line 1500, in visit_forwardref_type
    raise RuntimeError('Internal error: unresolved forward reference')
RuntimeError: Internal error: unresolved forward reference

The INTERNAL ERROR bunch of messages came out in the middle of the traceback. This is because the traceback is written to stdout but the other messages go to stderr.

The distinction between what goes where is intentional (see b20ee6ac21), but I forgot that stdout and stderr have separate buffers, and what happened here was that most of the traceback (there was much more of it) got flushed before stderr was flushed.

We can probably fix this by adding flush=True to all or most print() calls in the function.

Dominant language
Python
Stars
20.6k
Forks
3.3k
Avg merge
1d 18h
Merged PRs (30d)
54

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from python/mypy

All issues in python/mypy

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.