Pygments `Formatter.__init__` overloads do not work as intended
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 5.1k
- Forks
- 2.1k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 82
Description
In #6819 pygments.formatter.Formatter was made generic to improve the type safety:
_T = TypeVar("_T", str, bytes)
class Formatter(Generic[_T]):
@overload
def __init__(self: Formatter[str], *, encoding: None = ..., outencoding: None = ..., **options) -> None: ...
@overload
def __init__(self: Formatter[bytes], *, encoding: str, outencoding: None = ..., **options) -> None: ...
@overload
def __init__(self: Formatter[bytes], *, encoding: None = ..., outencoding: str, **options) -> None: ...
Apparently this was never tested, since it does not actually work:
from pygments.formatters.html import HtmlFormatter
reveal_type(HtmlFormatter())
reveal_type(HtmlFormatter(encoding='utf-8'))
$ mypy foo.py
foo.py:3: note: Revealed type is "pygments.formatters.html.HtmlFormatter[builtins.str*]"
foo.py:4: note: Revealed type is "pygments.formatters.html.HtmlFormatter[builtins.str*]"
$ pyright foo.py
/tmp/foo.py:3:13 - information: Type of "HtmlFormatter(encoding='utf-8')" is "HtmlFormatter[Unknown]"
/tmp/foo.py:4:13 - information: Type of "HtmlFormatter()" is "HtmlFormatter[Unknown]"
The intention of the overloads is that HtmlFormatter(encoding='utf-8') should be of type HtmlFormatter[builtins.bytes].
I am not sure if this is a mypy bug or if such overload inference should even work in the first place (given that pyright also infers the type variable to be Unknown: https://github.com/microsoft/pyright/issues/3146).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Formatter overloads introduced in #6819 and the HtmlFormatter entry point. Reproduce the two reveal_type examples with mypy and pyright, then determine whether the overload declarations or checker behavior prevents encoding='utf-8' from inferring HtmlFormatter[bytes]. Done means the intended inferred types are supported or the limitation is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100