'autosummary: failed to import' when using numpydoc instead of sphinx.ext.napoleon

I've been playing with doc build for a while now, using this sandbox repository with a simple project and code-base. I am running into an issue a bit similar to #80, I'm at a complete loss and I only managed to narrow it down to 'bad things happen when I used numpydoc on macOS and on Windows'.

When building the documentation on my Linux PC, or on the GitHub CI with this workflow, the build works without any issue. Here is the current version deployed.
When building the documentation locally on my Windows or macOS PC, I'm greeted with 52 similar warnings. You can find 2 of them below:

C:\Users\Mathieu\Documents\git\mscheltienne\simple-stimuli\stimuli\audio\noise.py:docstring of stimuli.audio.noise.Noise:36: WARNING: autosummary: failed to import color.
Possible hints:
* ModuleNotFoundError: No module named 'color'
* AttributeError: module 'stimuli.audio' has no attribute 'color'
* ModuleNotFoundError: No module named 'stimuli.audio.color'
* KeyError: 'color'
* AttributeError: module 'stimuli.audio.Noise' has no attribute 'color'
* ModuleNotFoundError: No module named 'stimuli.audio.Noise.color'; 'stimuli.audio.Noise' is not a package
* ValueError: not enough values to unpack (expected 2, got 1)
C:\Users\Mathieu\Documents\git\mscheltienne\simple-stimuli\stimuli\audio\noise.py:docstring of stimuli.audio.noise.Noise:36: WARNING: autosummary: failed to import duration.
Possible hints:
* ModuleNotFoundError: No module named 'stimuli.audio.duration'
* AttributeError: module 'stimuli.audio' has no attribute 'duration'
* ModuleNotFoundError: No module named 'stimuli.audio.Noise.duration'; 'stimuli.audio.Noise' is not a package
* ModuleNotFoundError: No module named 'duration'
* KeyError: 'duration'
* AttributeError: module 'stimuli.audio.Noise' has no attribute 'duration'
* ValueError: not enough values to unpack (expected 2, got 1)

And obviously, the documentation is a bit broken. Based on the pasted warnings, take the Noise class defined in noise.py: Left -> local (windows). Right -> CI (ubuntu). Both build on the same commit,

Why is numpydoc involved?

It turns out that if I remove numpydoc from the extensions and load sphinx.ext.napoleon instead, the doc builds without warnings (except the missing x-ref since they are not defined anymore). However, the render is missing the summary tables, so at least it makes sense that warnings about those tables are missing:

And the attributes and method tables that are failing are a feature of numpydoc, right?

In both cases, the generated .rst is the same:

stimuli.audio.Noise
===================

.. currentmodule:: stimuli.audio

.. autoclass:: Noise
   :members:
   :inherited-members:

I tried with different templates, and with the default templates, but nothing changed.

I'm at a complete loss as to why the doc build is failing on macOS or Windows, and not on Linux, I'm not even sure it's a bug, or an issue with my configuration. Here is a couple of lines to set-up the project if you want to give it a try:

git clone https://github.com/mscheltienne/simple-stimuli
python3 -m venv test --copies
source test/bin/activate
python3 -m pip install --upgrade pip setuptools wheel
cd simple-stimuli
python3 -m pip install -e ".[doc]"
cd doc
make html

Reproduce the documentation build using the linked simple-stimuli sandbox and the documented environment setup, comparing Linux or CI with Windows and macOS. Start with stimuli/audio/noise.py and the generated autosummary for stimuli.audio.Noise, then compare builds using numpydoc versus sphinx.ext.napoleon; done means the platform-specific import warnings and broken summary tables are resolved.