python / python/mypy

stubtest: More information in concise errors

Open
#15,011 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature topic-stubtest
Dominant language
Python
Stars
20.6k
Forks
3.3k
PR merge metrics
PR metrics pending

Description

Feature

The default error message is 5 lines:

  1. An error message
  2. Stub file/line context information
  3. Stub line contents
  4. Runtime file/line context information
  5. Runtime line contents

The concise error message is 1 line, but all it contains is the first point above.

Instead, it would be nice if it included context information, i.e., printing something like file:line:error message. Extra nice would be if it printed the message twice, once for stub and once for runtime (unless the error was that one was missing of course.)

Also, the note about unused allowlist entry includes no context information in either mode, but it would be nice if it did.

Pitch

In https://github.com/matplotlib/matplotlib/pull/25586, I am trying to get stubtest errors posted to PRs. Because the concise format contains no context information, I must use the verbose messages, but as those are multiple lines, it is difficult to easily parse those. In my test PR, you can see that I am able to catch 4 out of 6 errors. I cannot catch the allowlist notes because they include no context information. I cannot catch a missing stub entry because that error message includes no line number.

As you can see, I need to create a setup for each of the 5 lines. If the concise errors included context information, I could simplify that to just %f:%l:%m or similar.

Theoretically, this could also be useful for any editor that supported vim's errorformat.

Also cc'ing @hauntsaninja from https://github.com/matplotlib/matplotlib/pull/24976#issuecomment-1496899794

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at stubtest's concise-error formatting entry point and compare it with verbose output and allowlist-note handling. Trace how missing stub entries and runtime or stub locations are represented. Done means concise output provides parseable context for the reported cases, with coverage for the requested error and allowlist-note formats.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.