fedora-python / fedora-python/tox-current-env

0.0.8: pytest is failing in few units

Offen
#52 25 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
tests
Vorherrschende Sprache
Python
Sterne
26
Forks
9
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
- `python3 -sBm build -w --no-isolation`
- because I'm calling `build` with `--no-isolation` I'm using during all processes only locally installed modules
- install .whl file in
- run pytest with PYTHONPATH pointing to sitearch and sitelib inside

Here is pytest output:

```console
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-tox-current-env-0.0.8-4.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-tox-current-env-0.0.8-4.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.16, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8, configfile: tox.ini
plugins: xdist-3.1.0

[..]

sx.........................................s...s........................................................F.F.............FF..FF......F [100%]
================================================================================= FAILURES =================================================================================
____________________________________________________________ test_noquiet_installed_packages[--print-deps-to=-] ____________________________________________________________
[gw19] linux -- Python 3.8.16 /usr/bin/python3

flag = '--print-deps-to=-'

@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break

# default tox produces output sorted by package names
> assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0].lower()
)
E AssertionError: assert ['Babel==2.11...=2.13.0', ...] == ['alabaster==...==0.9.0', ...]
E At index 0 diff: 'Babel==2.11.0' != 'alabaster==0.7.12'
E Use -v to get more diff

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:619: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
py38 create: /tmp/pytest-of-tkloczko/pytest-487/popen-gw19/test_noquiet_installed_package0/projdir/.tox/py38
py38 installed: Babel==2.11.0,Brlapi==0.8.3,Jinja2==3.1.2,MarkupSafe==2.1.1,PyGObject==3.42.2,Pygments==2.13.0,Sphinx==5.3.0,alabaster==0.7.12,appdirs==1.4.4,attrs==22.1.0,build==0.9.0,charset-normalizer==3.0.1,cssselect==1.1.0,distlib==0.3.6,distro==1.8.0,dnspython==2.2.1,docutils==0.19,exceptiongroup==1.0.0,execnet==1.9.0,extras==1.0.0,filelock==3.8.0,fixtures==4.0.0,gpg==1.17.1-unknown,idna==3.4,imagesize==1.4.1,importlib-metadata==5.1.0,iniconfig==1.1.1,libcomps==0.1.19,lit==15.0.6,louis==3.24.0,lxml==4.9.1,meson==0.64.1,numpy==1.23.1,packaging==21.3,pbr==5.9.0,pep517==0.13.0,pip==22.3.1,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.9,pytest==7.2.0,pytest-xdist==3.1.0,python-dateutil==2.8.2,pytz==2022.4,requests==2.28.1,rpm==4.17.0,scour==0.38.2,setuptools==65.6.3,six==1.16.0,snowballstemmer==2.2.0,sphinxcontrib-applehelp==1.0.2.dev20221204,sphinxcontrib-devhelp==1.0.2.dev20221204,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1.dev20221204,sphinxcontrib-qthelp==1.0.3.dev20221204,sphinxcontrib-serializinghtml==1.1.5,testtools==2.5.0,toml==0.10.2,tomli==2.0.1,tox==3.26.0,tox-current-env==0.0.8,urllib3==1.26.12,virtualenv==20.17.0,wheel==0.38.4,zipp==3.11.0
py38 run-test-pre: PYTHONHASHSEED='3748779245'
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

______________________________________________________________ test_noquiet_installed_packages[--current-env] ______________________________________________________________
[gw23] linux -- Python 3.8.16 /usr/bin/python3

flag = '--current-env'

@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break

# default tox produces output sorted by package names
> assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0].lower()
)
E AssertionError: assert ['Babel==2.11...=2.13.0', ...] == ['alabaster==...==0.9.0', ...]
E At index 0 diff: 'Babel==2.11.0' != 'alabaster==0.7.12'
E Use -v to get more diff

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:619: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
py38 create: /tmp/pytest-of-tkloczko/pytest-487/popen-gw23/test_noquiet_installed_package0/projdir/.tox/py38
py38 installed: Babel==2.11.0,Brlapi==0.8.3,Jinja2==3.1.2,MarkupSafe==2.1.1,PyGObject==3.42.2,Pygments==2.13.0,Sphinx==5.3.0,alabaster==0.7.12,appdirs==1.4.4,attrs==22.1.0,build==0.9.0,charset-normalizer==3.0.1,cssselect==1.1.0,distlib==0.3.6,distro==1.8.0,dnspython==2.2.1,docutils==0.19,exceptiongroup==1.0.0,execnet==1.9.0,extras==1.0.0,filelock==3.8.0,fixtures==4.0.0,gpg==1.17.1-unknown,idna==3.4,imagesize==1.4.1,importlib-metadata==5.1.0,iniconfig==1.1.1,libcomps==0.1.19,lit==15.0.6,louis==3.24.0,lxml==4.9.1,meson==0.64.1,numpy==1.23.1,packaging==21.3,pbr==5.9.0,pep517==0.13.0,pip==22.3.1,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.9,pytest==7.2.0,pytest-xdist==3.1.0,python-dateutil==2.8.2,pytz==2022.4,requests==2.28.1,rpm==4.17.0,scour==0.38.2,setuptools==65.6.3,six==1.16.0,snowballstemmer==2.2.0,sphinxcontrib-applehelp==1.0.2.dev20221204,sphinxcontrib-devhelp==1.0.2.dev20221204,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1.dev20221204,sphinxcontrib-qthelp==1.0.3.dev20221204,sphinxcontrib-serializinghtml==1.1.5,testtools==2.5.0,toml==0.10.2,tomli==2.0.1,tox==3.26.0,tox-current-env==0.0.8,urllib3==1.26.12,virtualenv==20.17.0,wheel==0.38.4,zipp==3.11.0
py38 run-test-pre: PYTHONHASHSEED='1599822279'
py38 run-test: commands[0] | python -c 'import os, sys; print(os.path.realpath(sys.exec_prefix), "is the exec_prefix")'
/usr is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

__________________________________________________________________ test_noquiet_installed_packages[None] ___________________________________________________________________
[gw20] linux -- Python 3.8.16 /usr/bin/python3

flag = None

@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break

# default tox produces output sorted by package names
assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0].lower()
)

# without a flag, the output must match tox defaults
if not flag:
> assert len(packages) == 3
E AssertionError: assert 4 == 3
E + where 4 = len(['py==1.11.0', 'six==1.16.0', 'test @ file:///tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip', 'tox-current-env==0.0.8'])

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:625: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
GLOB sdist-make: /tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/setup.py
py38 create: /tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/py38
py38 installdeps: six, py
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38 inst: /tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38 installed: py==1.11.0,six==1.16.0,test @ file:///tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip,tox-current-env==0.0.8
py38 run-test-pre: PYTHONHASHSEED='3943385495'
py38 run-test: commands[0] | python -c 'import os, sys; print(os.path.realpath(sys.exec_prefix), "is the exec_prefix")'
/tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

______________________________________________________________________ test_regular_run_native_toxenv ______________________________________________________________________
[gw46] linux -- Python 3.8.16 /usr/bin/python3

def test_regular_run_native_toxenv():
result = tox("-e", NATIVE_TOXENV)
lines = sorted(result.stdout.splitlines()[:1])
> assert f"/.tox/{NATIVE_TOXENV} is the exec_prefix" in lines[0]
E assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:479: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/tmp/pytest-of-tkloczko/pytest-487/popen-gw46/test_regular_run_native_toxenv0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

_________________________________________________ test_regular_after_first_print_deps_is_supported[--print-deps-to-file=-] _________________________________________________
[gw16] linux -- Python 3.8.16 /usr/bin/python3

print_deps_stdout_arg = '--print-deps-to-file=-'

def test_regular_after_first_print_deps_is_supported(print_deps_stdout_arg):
result = tox("-e", NATIVE_TOXENV, print_deps_stdout_arg)
assert result.stdout.splitlines()[0] == "six"
result = tox("-e", NATIVE_TOXENV)
lines = sorted(result.stdout.splitlines()[:1])
assert "--recreate" not in result.stderr
> assert f"/.tox/{NATIVE_TOXENV} is the exec_prefix" in lines[0]
E assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:514: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/tmp/pytest-of-tkloczko/pytest-487/popen-gw16/test_regular_after_first_print0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

___________________________________________________ test_regular_after_first_print_deps_is_supported[--print-deps-to=-] ____________________________________________________
[gw14] linux -- Python 3.8.16 /usr/bin/python3

print_deps_stdout_arg = '--print-deps-to=-'

def test_regular_after_first_print_deps_is_supported(print_deps_stdout_arg):
result = tox("-e", NATIVE_TOXENV, print_deps_stdout_arg)
assert result.stdout.splitlines()[0] == "six"
result = tox("-e", NATIVE_TOXENV)
lines = sorted(result.stdout.splitlines()[:1])
assert "--recreate" not in result.stderr
> assert f"/.tox/{NATIVE_TOXENV} is the exec_prefix" in lines[0]
E assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:514: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/tmp/pytest-of-tkloczko/pytest-487/popen-gw14/test_regular_after_first_print0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

___________________________________________________ test_regular_after_first_print_deps_is_supported[--print-deps-only] ____________________________________________________
[gw3] linux -- Python 3.8.16 /usr/bin/python3

print_deps_stdout_arg = '--print-deps-only'

def test_regular_after_first_print_deps_is_supported(print_deps_stdout_arg):
result = tox("-e", NATIVE_TOXENV, print_deps_stdout_arg)
assert result.stdout.splitlines()[0] == "six"
result = tox("-e", NATIVE_TOXENV)
lines = sorted(result.stdout.splitlines()[:1])
assert "--recreate" not in result.stderr
> assert f"/.tox/{NATIVE_TOXENV} is the exec_prefix" in lines[0]
E assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:514: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/tmp/pytest-of-tkloczko/pytest-487/popen-gw3/test_regular_after_first_print0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_integration.py:136: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:117: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:459: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
XFAIL tests/test_integration.py::test_allenvs_print_deps_to_file_print_extras_to_other_file[True-True] - reason: Unsupported combination of parameters
FAILED tests/test_integration.py::test_noquiet_installed_packages[--print-deps-to=-] - AssertionError: assert ['Babel==2.11...=2.13.0', ...] == ['alabaster==...==0.9.0', ...]
FAILED tests/test_integration.py::test_noquiet_installed_packages[--current-env] - AssertionError: assert ['Babel==2.11...=2.13.0', ...] == ['alabaster==...==0.9.0', ...]
FAILED tests/test_integration.py::test_noquiet_installed_packages[None] - AssertionError: assert 4 == 3
FAILED tests/test_integration.py::test_regular_run_native_toxenv - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-to-file=-] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-to=-] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-only] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
=========================================================== 7 failed, 122 passed, 3 skipped, 1 xfailed in 16.99s ===========================================================
```

Here is list of installed modules in build env

```console
Package Version
----------------------------- -----------------
alabaster 0.7.12
appdirs 1.4.4
attrs 22.1.0
Babel 2.11.0
Brlapi 0.8.3
build 0.9.0
charset-normalizer 3.0.1
cssselect 1.1.0
distlib 0.3.6
distro 1.8.0
dnspython 2.2.1
docutils 0.19
exceptiongroup 1.0.0
execnet 1.9.0
extras 1.0.0
filelock 3.8.0
fixtures 4.0.0
gpg 1.17.1-unknown
idna 3.4
imagesize 1.4.1
importlib-metadata 5.1.0
iniconfig 1.1.1
Jinja2 3.1.2
libcomps 0.1.19
louis 3.24.0
lxml 4.9.1
MarkupSafe 2.1.1
numpy 1.23.1
packaging 21.3
pbr 5.9.0
pep517 0.13.0
pip 22.3.1
platformdirs 2.5.2
pluggy 1.0.0
py 1.11.0
Pygments 2.13.0
PyGObject 3.42.2
pyparsing 3.0.9
pytest 7.2.0
pytest-xdist 3.1.0
python-dateutil 2.8.2
pytz 2022.4
requests 2.28.1
rpm 4.17.0
scour 0.38.2
setuptools 65.6.3
six 1.16.0
snowballstemmer 2.2.0
Sphinx 5.3.0
sphinxcontrib-applehelp 1.0.2.dev20221204
sphinxcontrib-devhelp 1.0.2.dev20221204
sphinxcontrib-htmlhelp 2.0.0
sphinxcontrib-jsmath 1.0.1.dev20221204
sphinxcontrib-qthelp 1.0.3.dev20221204
sphinxcontrib-serializinghtml 1.1.5
testtools 2.5.0
toml 0.10.2
tomli 2.0.1
tox 3.26.0
urllib3 1.26.12
virtualenv 20.17.0
wheel 0.38.4
zipp 3.11.0
```

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Rechercherichtung

Start with tests/test_integration.py, especially test_noquiet_installed_packages and test_regular_run_native_toxenv, and reproduce the reported pytest -ra run after the PEP517 wheel install with PYTHONPATH set to the install prefix. Compare the failing package ordering, package count, and exec_prefix assertions with the captured output. Done means the affected integration tests pass in the reported non-root packaging workflow.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
build-system, testing-qa
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.