pytest-dev / pytest-dev/pytest

Duplicate, conflicting imports due to changes with editable installs (Migrating from setup.py to pyproject.toml may subtly require enabling namespace support)

Open
#13,324 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

topic: collection
Dominant language
Python
Stars
14.5k
Forks
3.4k
Avg merge
2d 9h
Merged PRs (30d)
35

Description

  • a detailed description of the bug or problem you are having

We've converted an older setuptools/setup.py based project to uv and pyproject.toml. The editable installation (using uv sync) uses this requirements.txt snippet:

-e /srv/s-dev/fc.directory[test]
    # via -r requirements.in
-e /srv/s-dev/fc.support
    # via
    #   -r requirements.in

This ends up with the following sys.path when starting the venv's interpreter:

['',
 '/nix/store/80v02bzfz0g507q8b1g9ylmygc572hzd-python3-3.12.7/lib/python312.zip',
 '/nix/store/80v02bzfz0g507q8b1g9ylmygc572hzd-python3-3.12.7/lib/python3.12',
 '/nix/store/80v02bzfz0g507q8b1g9ylmygc572hzd-python3-3.12.7/lib/python3.12/lib-dynload',
 '/srv/s-dev/deployment/work/directorybuild/.venv/lib/python3.12/site-packages',
 '__editable__.fc_directory-2.69.1.dev0.finder.__path_hook__',
 '/srv/s-dev/fc.support/src']

Now, in the actual application this works fine. Howevever, our tests fail because modules are imported twice and I can see that within the pytest environment the path for the namespace package appears a second time
The first dependency is added using a finder hook (something setuptools introduced a while ago it appears).

The second interestingly uses a plain .pth file and ends up working fine.

Here's the path from the pytest session:

(Pdb) pp sys.path
['/srv/s-dev/fc.directory/src/fc',
 '/srv/s-dev/deployment/work/directorybuild/.venv/bin',
 '/nix/store/80v02bzfz0g507q8b1g9ylmygc572hzd-python3-3.12.7/lib/python312.zip',
 '/nix/store/80v02bzfz0g507q8b1g9ylmygc572hzd-python3-3.12.7/lib/python3.12',
 '/nix/store/80v02bzfz0g507q8b1g9ylmygc572hzd-python3-3.12.7/lib/python3.12/lib-dynload',
 '/srv/s-dev/deployment/work/directorybuild/.venv/lib/python3.12/site-packages',
 '__editable__.fc_directory-2.69.1.dev0.finder.__path_hook__',
 '/srv/s-dev/fc.support/src',
 '/srv/s-dev/deployment/work/directorybuild/.venv/lib/python3.12/site-packages/setuptools/_vendor']
  • output of pip list from the virtual environment you are using
Package               Version      Editable project location
--------------------- ------------ -------------------------
alembic               1.13.2
annotated-types       0.6.0
anyio                 4.4.0
apipkg                3.0.2
arrow                 1.2.3
attrs                 22.2.0
beaker                1.13.0
beautifulsoup4        4.12.3
black                 24.8.0
certifi               2024.7.4
cffi                  1.17.1
chameleon             4.5.4
chardet               4.0.0
charset-normalizer    2.0.12
click                 8.1.8
colander              2.0
colorama              0.4.6
consulate-fc-nix-test 1.1.0a1
coverage              7.2.7
cryptography          43.0.3
cssselect2            0.7.0
dataproperty          0.55.1
decorator             4.4.2
deform                2.0.8
defusedxml            0.7.1
deprecation           2.1.0
ecdsa                 0.19.1
exceptiongroup        1.1.2
execnet               1.7.1
fanstatic             1.5
fastapi               0.115.11
fc-directory          2.69.1.dev0  /srv/s-dev/fc.directory
fc-support            1.0.dev0     /srv/s-dev/fc.support
future                1.0.0
gocept-cache          6.0
gocept-collmex        2.1.0
gocept-testdb         6.0
gocept-testing        4.0
greenlet              3.0.3
grokcore-component    4.1.1
h11                   0.14.0
httpcore              0.16.3
httpx                 0.23.3
humanfriendly         4.18
hupper                1.12.1
icalendar             4.0.4
idna                  3.7
iniconfig             2.0.0
iso8601               0.1.16
jinja2                3.1.6
jwcrypto              1.5.6
lxml                  5.3.1
mako                  1.2.4
markdown              3.4.1
markdown2             2.5.3
markupsafe            2.1.2
martian               1.3.post1
mbstrdecoder          1.1.2
mock                  3.0.5
mypy-extensions       1.0.0
nagiosplugin          1.3.3
netaddr               0.7.19
oic                   1.6.1
outcome               1.2.0
packaging             23.0
passlib               1.7.4
paste                 3.10.1
pastedeploy           3.1.0
pathspec              0.9.0
pathvalidate          2.5.2
peppercorn            0.6
perfmetrics           3.3.0
pillow                10.4.0
plaster               1.1.2
plaster-pastedeploy   0.7
platformdirs          4.2.2
plone-testing         7.0.3
pluggy                1.5.0
pmw-py3               2.1
preppy                4.2.2
psycopg2              2.9.10
pyasn1                0.4.8
pyasn1-modules        0.2.7
pycparser             2.19
pycryptodomex         3.20.0
pydantic              2.6.3
pydantic-core         2.16.3
pydantic-settings     2.2.1
pygments              2.16.1
pyjwkest              1.4.2
pyotp                 2.3.0
pypdf                 4.3.1
pyramid               1.10.8
pyramid-beaker        0.8
pyramid-chameleon     0.3
pyramid-debugtoolbar  4.5.1
pyramid-deform        0.2
pyramid-fanstatic     0.6.dev0+fc0
pyramid-mailer        0.15.1
pyramid-mako          1.1.0
pyramid-retry         2.1
pyramid-rpc           0.8
pyramid-tm            2.6
pysocks               1.7.1
pyst2                 0.5.1
pytablewriter         0.64.2
pytest                8.3.2
pytest-cache          1.0
pytest-cov            4.1.0
pytest-localserver    0.8.0
pytest-rerunfailures  12.0
pytest-timeout        2.1.0
pytest-xdist          3.3.1
python-dateutil       2.8.2
python-dotenv         1.0.1
python-keycloak       3.12.0
python-ldap           3.4.3
pytz                  2024.2
pyyaml                6.0.2
redis                 4.5.5
reportlab             4.2.2
repoze-lru            0.7
repoze-sendmail       4.4.1
requests              2.32.3
requests-toolbelt     1.0.0
rfc3986               1.5.0
rlextra               4.2.2
selenium              4.10.0
setuptools            74.0.0
shortuuid             0.5.0
simplejson            3.18.3
six                   1.16.0
slowlog               0.9
sniffio               1.3.0
sortedcontainers      2.4.0
soupsieve             1.9.6
sqlalchemy            2.0.35
sshpubkeys            3.1.0
starlette             0.46.1
structlog             20.1.0
svglib                1.4.1
systemd-python        235
tabledata             1.3.1
tabulate              0.9.0
tcolorpy              0.1.2
tinycss2              1.2.1
transaction           3.0.1
translationstring     1.3
trio                  0.22.2
trio-websocket        0.10.3
typepy                1.3.0
typing-extensions     4.10.0
urllib3               1.26.19
validate-email        1.3
venusian              3.1.0
waitress              2.1.2
webencodings          0.5.1
webob                 1.8.8
webtest               3.0.0
werkzeug              3.0.3
wsgiproxy2            0.4.6
wsproto               1.2.0
xlsxwriter            1.2.6
zope-cachedescriptors 5.0
zope-component        6.0
zope-configuration    5.0.1
zope-deprecation      5.0
zope-dottedname       6.0
zope-event            5.0
zope-hookable         6.0
zope-i18nmessageid    6.1.0
zope-interface        7.0.3
zope-lifecycleevent   5.0
zope-location         5.0
zope-proxy            5.3
zope-schema           7.0.1
zope-security         7.1
zope-sendmail         6.2
zope-sqlalchemy       3.1
zope-testbrowser      7.0
zope-testing          5.0.1
  • pytest and operating system versions
platform linux -- Python 3.12.7, pytest-8.3.2, pluggy-1.5.0
DISTRIB_CODENAME=uakari
DISTRIB_DESCRIPTION="NixOS 24.05 (Uakari)"
DISTRIB_ID=nixos
DISTRIB_RELEASE="24.05"
LSB_VERSION="24.05 (Uakari)"
  • minimal example if possible

Sorry, no idea. I'm still trying to find a workaround for me ATM. Here's a number of issues I've been stumbling into and they seem to be pointing towards "client code needs to follow" as the answer ...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No project file or failing test is named. Start by reproducing the pytest session with the shown Python 3.12.7 environment and editable installations, then compare the namespace package paths from the setuptools finder hook and the .pth entry. Done means the duplicate import behavior is reproduced and a focused pytest regression test demonstrates the corrected path handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.