pybind / pybind/pybind11-stubgen
Using `KeysView`/`ValuesView`/`ItemsView` across sub modules can result in "can't find/import" error
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 361
- Forks
- 75
- PR merge metrics
- No merged PRs in 30d
Description
Consider the following example:
#include <functional>
#include <unordered_map>
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <pybind11/stl_bind.h>
namespace py = pybind11;
struct MyKeyA
{
bool operator==(const MyKeyA&) const = default;
};
struct MyKeyB
{
bool operator==(const MyKeyB&) const = default;
};
template <>
struct std::hash<MyKeyA>
{
size_t operator()(const MyKeyA& obj) const
{
return 0; // omitted.
}
};
template <>
struct std::hash<MyKeyB>
{
size_t operator()(const MyKeyB& obj) const
{
return 0; // omitted.
}
};
PYBIND11_MODULE(my_native_module, m)
{
py::module_ module_a = m.def_submodule("module_a");
py::class_<MyKeyA>(module_a, "MyKeyA");
py::bind_map<std::unordered_map<MyKeyA, int>>(module_a, "MapAToInt"); // (1)
py::module_ module_b = m.def_submodule("module_b");
py::class_<MyKeyB>(module_b, "MyKeyB");
py::bind_map<std::unordered_map<MyKeyA, bool>>(module_b, "MapAToBool"); // (2)
py::bind_map<std::unordered_map<MyKeyB, bool>>(module_b, "MapBToBool");
}
The pattern here is:
- There are two different key types
KeyAandKeyB, one defined in sub-modulemodule_athe other inmodule_b. module_adefines a mapdict[KeyA, int]in line (1)module_bdefines two maps,dict[KeyA, bool]anddict[KeyB, bool].
Trying to generate stubs for the module fails with:
pybind11_stubgen - [ ERROR] In my_native_module.module_b.MapAToBool.keys : Can't find/import 'my_native_module.module_a.KeysView'
pybind11_stubgen - [ INFO] Terminating due to previous errors
The issue seems to be triggered by the co-existence of lines (1) and (2), i.e., if KeysA gets used in module_a in a map, it somehow "moves the KeysView[...] type over" into that module. By commenting out line (1), the KeysView ends up in the other sub-module which works. Conversely, commenting out line (2) also works, because then module_b is not trying to reference the invalid KeysView path.
For comparison, when commenting out line (1) the symbols in submodule a/b are divided as (trimmed outputs of dir(my_native_module.module_X)):
# Symbols in module_a:
MyKeyA
# Symbols in module_b:
ItemsView[my_native_module.module_a.MyKeyA, bool]
ItemsView[my_native_module.module_b.MyKeyB, bool]
KeysView[my_native_module.module_a.MyKeyA]
KeysView[my_native_module.module_b.MyKeyB]
MapAToBool
MapBToBool
MyKeyB
ValuesView[bool]
Note that all the KeysView are local to b. Commenting in line (1) "moves" the KeysView into module_a leading to the problematic module path:
# Symbols in module_a:
ItemsView[my_native_module.module_a.MyKeyA, int]
KeysView[my_native_module.module_a.MyKeyA]
MapAToInt
MyKeyA
ValuesView[int]
# Symbols in module_b:
ItemsView[my_native_module.module_a.MyKeyA, bool]
ItemsView[my_native_module.module_b.MyKeyB, bool]
KeysView[my_native_module.module_b.MyKeyB]
MapAToBool
MapBToBool
MyKeyB
ValuesView[bool]
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the provided C++ reproduction using two submodules and inspect how generated KeysView, ValuesView, and ItemsView symbols are assigned to module paths. Confirm the failure with pybind11-stubgen; done when the example generates stubs without trying to import KeysView from the wrong submodule.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100