pybind / pybind/pybind11-stubgen

Enum with custom `__str__` results in non-type-checkable stub

Open
#207 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
361
Forks
75
PR merge metrics
No merged PRs in 30d

Description

The following is a minimal example involving an enum with a custom __str__ implementation:

#include <stdexcept>
#include <string>

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

namespace py = pybind11;

enum class MyEnum
{
  FOO,
  BAR,
};

std::string customToString(MyEnum value)
{
  switch (value) {
    case MyEnum::FOO:
      return "foo";
    case MyEnum::BAR:
      return "bar";
    default:
      throw std::invalid_argument("Invalid value");
  }
}

PYBIND11_MODULE(my_native_module, m)
{
  pybind11::enum_<MyEnum>(m, "MyEnum")
      .value("FOO", MyEnum::FOO)
      .value("BAR", MyEnum::BAR)
      .def("__str__", &customToString);
}

Running the stub generator on this module produces:

from __future__ import annotations
import typing
__all__ = ['MyEnum']
class MyEnum:
    """
    Members:
    
      FOO
    
      BAR
    """
    BAR: typing.ClassVar[MyEnum]  # value = <MyEnum.BAR: 1>
    FOO: typing.ClassVar[MyEnum]  # value = <MyEnum.FOO: 0>
    __members__: typing.ClassVar[dict[str, MyEnum]]  # value = {'FOO': <MyEnum.FOO: 0>, 'BAR': <MyEnum.BAR: 1>}
    @staticmethod
    def name(*args, **kwargs):
        """
        __str__(*args, **kwargs)
        Overloaded function.
        
        1. __str__(self: handle) -> str
        
        2. __str__(self: my_native_module.MyEnum) -> str
        """
    def __eq__(self, other: typing.Any) -> bool:
        ...
    def __getstate__(self) -> int:
        ...
    def __hash__(self) -> int:
        ...
    def __index__(self) -> int:
        ...
    def __init__(self, value: int) -> None:
        ...
    def __int__(self) -> int:
        ...
    def __ne__(self, other: typing.Any) -> bool:
        ...
    def __repr__(self) -> str:
        ...
    def __setstate__(self, state: int) -> None:
        ...
    @typing.overload
    def __str__(self) -> str:
        ...
    @typing.overload
    def __str__(self) -> str:
        ...
    @property
    def value(self) -> int:
        ...

The issue here is that mypy isn't able to type check this stub because of the redundant overload, i.e., trying to use the stub with mypy errors with:

out/my_native_module.pyi:47: error: Overloaded function signature 2 will never be matched: signature 1's parameter type(s) are the same or broader  [misc]
Found 1 error in 1 file (checked 1 source file)

This issue seems to be related to https://github.com/pybind/pybind11/issues/4585 because there is also something funny happening related to name (which should be a property, but somehow adding a custom __str__ switches it to a method instead -- it is actually an instancemethod not a staticmethod though). Inspecting the underlying annotation produced by pybind11 at runtime:

In [1]: import my_native_module

In [2]: print(my_native_module.MyEnum.__str__.__doc__)
__str__(*args, **kwargs)
Overloaded function.

1. __str__(self: handle) -> str

2. __str__(self: my_native_module.MyEnum) -> str


In [3]: print(my_native_module.MyEnum.name.__doc__)
__str__(*args, **kwargs)
Overloaded function.

1. __str__(self: handle) -> str

2. __str__(self: my_native_module.MyEnum) -> str

So in a sense the root cause is the broken annotation produced by pybind11 here, but perhaps it is an easy fix on the stub generator side to omit such redundant overloads as a robustification? (The mypy stubgen just seems to ignore them, and therefore still produces type-checkable output in this case.)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the generated my_native_module.pyi from the enum example and run mypy against it. Inspect how the stub generator handles the runtime str annotation and the generated name member. Done means the redundant str overloads are omitted or corrected so the generated stub passes mypy type checking.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
developer-experience, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.