fedora-python / fedora-python/tox-current-env

0.0.8: pytest is failing in few units

オープン
#52 コメント 25 件 リアクション 0 件 担当者 0 名 GitHub で見る
tests
主要言語
Python
スター
26
フォーク
9
PR マージ指標
30日以内にマージされた PR はありません

説明

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.
- `python3 -sBm build -w --no-isolation`
- because I'm calling `build` with `--no-isolation` I'm using during all processes only locally installed modules
- install .whl file in
- run pytest with PYTHONPATH pointing to sitearch and sitelib inside

Here is pytest output:

```console
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-tox-current-env-0.0.8-4.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-tox-current-env-0.0.8-4.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.16, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8, configfile: tox.ini
plugins: xdist-3.1.0

[..]

sx.........................................s...s........................................................F.F.............FF..FF......F [100%]
================================================================================= FAILURES =================================================================================
____________________________________________________________ test_noquiet_installed_packages[--print-deps-to=-] ____________________________________________________________
[gw19] linux -- Python 3.8.16 /usr/bin/python3

flag = '--print-deps-to=-'

@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break

# default tox produces output sorted by package names
> assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0].lower()
)
E AssertionError: assert ['Babel==2.11...=2.13.0', ...] == ['alabaster==...==0.9.0', ...]
E At index 0 diff: 'Babel==2.11.0' != 'alabaster==0.7.12'
E Use -v to get more diff

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:619: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
py38 create: /tmp/pytest-of-tkloczko/pytest-487/popen-gw19/test_noquiet_installed_package0/projdir/.tox/py38
py38 installed: Babel==2.11.0,Brlapi==0.8.3,Jinja2==3.1.2,MarkupSafe==2.1.1,PyGObject==3.42.2,Pygments==2.13.0,Sphinx==5.3.0,alabaster==0.7.12,appdirs==1.4.4,attrs==22.1.0,build==0.9.0,charset-normalizer==3.0.1,cssselect==1.1.0,distlib==0.3.6,distro==1.8.0,dnspython==2.2.1,docutils==0.19,exceptiongroup==1.0.0,execnet==1.9.0,extras==1.0.0,filelock==3.8.0,fixtures==4.0.0,gpg==1.17.1-unknown,idna==3.4,imagesize==1.4.1,importlib-metadata==5.1.0,iniconfig==1.1.1,libcomps==0.1.19,lit==15.0.6,louis==3.24.0,lxml==4.9.1,meson==0.64.1,numpy==1.23.1,packaging==21.3,pbr==5.9.0,pep517==0.13.0,pip==22.3.1,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.9,pytest==7.2.0,pytest-xdist==3.1.0,python-dateutil==2.8.2,pytz==2022.4,requests==2.28.1,rpm==4.17.0,scour==0.38.2,setuptools==65.6.3,six==1.16.0,snowballstemmer==2.2.0,sphinxcontrib-applehelp==1.0.2.dev20221204,sphinxcontrib-devhelp==1.0.2.dev20221204,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1.dev20221204,sphinxcontrib-qthelp==1.0.3.dev20221204,sphinxcontrib-serializinghtml==1.1.5,testtools==2.5.0,toml==0.10.2,tomli==2.0.1,tox==3.26.0,tox-current-env==0.0.8,urllib3==1.26.12,virtualenv==20.17.0,wheel==0.38.4,zipp==3.11.0
py38 run-test-pre: PYTHONHASHSEED='3748779245'
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

______________________________________________________________ test_noquiet_installed_packages[--current-env] ______________________________________________________________
[gw23] linux -- Python 3.8.16 /usr/bin/python3

flag = '--current-env'

@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break

# default tox produces output sorted by package names
> assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0].lower()
)
E AssertionError: assert ['Babel==2.11...=2.13.0', ...] == ['alabaster==...==0.9.0', ...]
E At index 0 diff: 'Babel==2.11.0' != 'alabaster==0.7.12'
E Use -v to get more diff

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:619: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
py38 create: /tmp/pytest-of-tkloczko/pytest-487/popen-gw23/test_noquiet_installed_package0/projdir/.tox/py38
py38 installed: Babel==2.11.0,Brlapi==0.8.3,Jinja2==3.1.2,MarkupSafe==2.1.1,PyGObject==3.42.2,Pygments==2.13.0,Sphinx==5.3.0,alabaster==0.7.12,appdirs==1.4.4,attrs==22.1.0,build==0.9.0,charset-normalizer==3.0.1,cssselect==1.1.0,distlib==0.3.6,distro==1.8.0,dnspython==2.2.1,docutils==0.19,exceptiongroup==1.0.0,execnet==1.9.0,extras==1.0.0,filelock==3.8.0,fixtures==4.0.0,gpg==1.17.1-unknown,idna==3.4,imagesize==1.4.1,importlib-metadata==5.1.0,iniconfig==1.1.1,libcomps==0.1.19,lit==15.0.6,louis==3.24.0,lxml==4.9.1,meson==0.64.1,numpy==1.23.1,packaging==21.3,pbr==5.9.0,pep517==0.13.0,pip==22.3.1,platformdirs==2.5.2,pluggy==1.0.0,py==1.11.0,pyparsing==3.0.9,pytest==7.2.0,pytest-xdist==3.1.0,python-dateutil==2.8.2,pytz==2022.4,requests==2.28.1,rpm==4.17.0,scour==0.38.2,setuptools==65.6.3,six==1.16.0,snowballstemmer==2.2.0,sphinxcontrib-applehelp==1.0.2.dev20221204,sphinxcontrib-devhelp==1.0.2.dev20221204,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1.dev20221204,sphinxcontrib-qthelp==1.0.3.dev20221204,sphinxcontrib-serializinghtml==1.1.5,testtools==2.5.0,toml==0.10.2,tomli==2.0.1,tox==3.26.0,tox-current-env==0.0.8,urllib3==1.26.12,virtualenv==20.17.0,wheel==0.38.4,zipp==3.11.0
py38 run-test-pre: PYTHONHASHSEED='1599822279'
py38 run-test: commands[0] | python -c 'import os, sys; print(os.path.realpath(sys.exec_prefix), "is the exec_prefix")'
/usr is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

__________________________________________________________________ test_noquiet_installed_packages[None] ___________________________________________________________________
[gw20] linux -- Python 3.8.16 /usr/bin/python3

flag = None

@pytest.mark.parametrize("flag", [None, "--print-deps-to=-", "--current-env"])
def test_noquiet_installed_packages(flag):
flags = (flag,) if flag else ()
result = tox("-e", NATIVE_TOXENV, *flags, quiet=False, check=False)
assert f"\n{NATIVE_TOXENV} installed: " in result.stdout
for line in result.stdout.splitlines():
if line.startswith(f"{NATIVE_TOXENV} installed: "):
packages = line.rpartition(" installed: ")[-1].split(",")
break

# default tox produces output sorted by package names
assert packages == sorted(
packages, key=lambda p: p.partition("==")[0].partition(" @ ")[0].lower()
)

# without a flag, the output must match tox defaults
if not flag:
> assert len(packages) == 3
E AssertionError: assert 4 == 3
E + where 4 = len(['py==1.11.0', 'six==1.16.0', 'test @ file:///tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip', 'tox-current-env==0.0.8'])

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:625: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
GLOB sdist-make: /tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/setup.py
py38 create: /tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/py38
py38 installdeps: six, py
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38 inst: /tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38 installed: py==1.11.0,six==1.16.0,test @ file:///tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/.tmp/package/1/test-0.0.0.zip,tox-current-env==0.0.8
py38 run-test-pre: PYTHONHASHSEED='3943385495'
py38 run-test: commands[0] | python -c 'import os, sys; print(os.path.realpath(sys.exec_prefix), "is the exec_prefix")'
/tmp/pytest-of-tkloczko/pytest-487/popen-gw20/test_noquiet_installed_package0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

______________________________________________________________________ test_regular_run_native_toxenv ______________________________________________________________________
[gw46] linux -- Python 3.8.16 /usr/bin/python3

def test_regular_run_native_toxenv():
result = tox("-e", NATIVE_TOXENV)
lines = sorted(result.stdout.splitlines()[:1])
> assert f"/.tox/{NATIVE_TOXENV} is the exec_prefix" in lines[0]
E assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:479: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/tmp/pytest-of-tkloczko/pytest-487/popen-gw46/test_regular_run_native_toxenv0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

_________________________________________________ test_regular_after_first_print_deps_is_supported[--print-deps-to-file=-] _________________________________________________
[gw16] linux -- Python 3.8.16 /usr/bin/python3

print_deps_stdout_arg = '--print-deps-to-file=-'

def test_regular_after_first_print_deps_is_supported(print_deps_stdout_arg):
result = tox("-e", NATIVE_TOXENV, print_deps_stdout_arg)
assert result.stdout.splitlines()[0] == "six"
result = tox("-e", NATIVE_TOXENV)
lines = sorted(result.stdout.splitlines()[:1])
assert "--recreate" not in result.stderr
> assert f"/.tox/{NATIVE_TOXENV} is the exec_prefix" in lines[0]
E assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:514: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/tmp/pytest-of-tkloczko/pytest-487/popen-gw16/test_regular_after_first_print0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

___________________________________________________ test_regular_after_first_print_deps_is_supported[--print-deps-to=-] ____________________________________________________
[gw14] linux -- Python 3.8.16 /usr/bin/python3

print_deps_stdout_arg = '--print-deps-to=-'

def test_regular_after_first_print_deps_is_supported(print_deps_stdout_arg):
result = tox("-e", NATIVE_TOXENV, print_deps_stdout_arg)
assert result.stdout.splitlines()[0] == "six"
result = tox("-e", NATIVE_TOXENV)
lines = sorted(result.stdout.splitlines()[:1])
assert "--recreate" not in result.stderr
> assert f"/.tox/{NATIVE_TOXENV} is the exec_prefix" in lines[0]
E assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:514: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/tmp/pytest-of-tkloczko/pytest-487/popen-gw14/test_regular_after_first_print0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

___________________________________________________ test_regular_after_first_print_deps_is_supported[--print-deps-only] ____________________________________________________
[gw3] linux -- Python 3.8.16 /usr/bin/python3

print_deps_stdout_arg = '--print-deps-only'

def test_regular_after_first_print_deps_is_supported(print_deps_stdout_arg):
result = tox("-e", NATIVE_TOXENV, print_deps_stdout_arg)
assert result.stdout.splitlines()[0] == "six"
result = tox("-e", NATIVE_TOXENV)
lines = sorted(result.stdout.splitlines()[:1])
assert "--recreate" not in result.stderr
> assert f"/.tox/{NATIVE_TOXENV} is the exec_prefix" in lines[0]
E assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."

/home/tkloczko/rpmbuild/BUILD/tox-current-env-0.0.8/tests/test_integration.py:514: AssertionError
--------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------
six
py
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
/tmp/pytest-of-tkloczko/pytest-487/popen-gw3/test_regular_after_first_print0/projdir/.tox/py38 is the exec_prefix
___________________________________ summary ____________________________________
py38: commands succeeded
congratulations :)

--------------------------------------------------------------------------- Captured stderr call ---------------------------------------------------------------------------

========================================================================= short test summary info ==========================================================================
SKIPPED [1] tests/test_integration.py:136: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:117: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
SKIPPED [1] tests/test_integration.py:459: This test needs python3.6, 3.7, 3.8, 3.9 and 3.10 available in $PATH
XFAIL tests/test_integration.py::test_allenvs_print_deps_to_file_print_extras_to_other_file[True-True] - reason: Unsupported combination of parameters
FAILED tests/test_integration.py::test_noquiet_installed_packages[--print-deps-to=-] - AssertionError: assert ['Babel==2.11...=2.13.0', ...] == ['alabaster==...==0.9.0', ...]
FAILED tests/test_integration.py::test_noquiet_installed_packages[--current-env] - AssertionError: assert ['Babel==2.11...=2.13.0', ...] == ['alabaster==...==0.9.0', ...]
FAILED tests/test_integration.py::test_noquiet_installed_packages[None] - AssertionError: assert 4 == 3
FAILED tests/test_integration.py::test_regular_run_native_toxenv - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-to-file=-] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-to=-] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
FAILED tests/test_integration.py::test_regular_after_first_print_deps_is_supported[--print-deps-only] - assert '/.tox/py38 is the exec_prefix' in "WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration."
=========================================================== 7 failed, 122 passed, 3 skipped, 1 xfailed in 16.99s ===========================================================
```

Here is list of installed modules in build env

```console
Package Version
----------------------------- -----------------
alabaster 0.7.12
appdirs 1.4.4
attrs 22.1.0
Babel 2.11.0
Brlapi 0.8.3
build 0.9.0
charset-normalizer 3.0.1
cssselect 1.1.0
distlib 0.3.6
distro 1.8.0
dnspython 2.2.1
docutils 0.19
exceptiongroup 1.0.0
execnet 1.9.0
extras 1.0.0
filelock 3.8.0
fixtures 4.0.0
gpg 1.17.1-unknown
idna 3.4
imagesize 1.4.1
importlib-metadata 5.1.0
iniconfig 1.1.1
Jinja2 3.1.2
libcomps 0.1.19
louis 3.24.0
lxml 4.9.1
MarkupSafe 2.1.1
numpy 1.23.1
packaging 21.3
pbr 5.9.0
pep517 0.13.0
pip 22.3.1
platformdirs 2.5.2
pluggy 1.0.0
py 1.11.0
Pygments 2.13.0
PyGObject 3.42.2
pyparsing 3.0.9
pytest 7.2.0
pytest-xdist 3.1.0
python-dateutil 2.8.2
pytz 2022.4
requests 2.28.1
rpm 4.17.0
scour 0.38.2
setuptools 65.6.3
six 1.16.0
snowballstemmer 2.2.0
Sphinx 5.3.0
sphinxcontrib-applehelp 1.0.2.dev20221204
sphinxcontrib-devhelp 1.0.2.dev20221204
sphinxcontrib-htmlhelp 2.0.0
sphinxcontrib-jsmath 1.0.1.dev20221204
sphinxcontrib-qthelp 1.0.3.dev20221204
sphinxcontrib-serializinghtml 1.1.5
testtools 2.5.0
toml 0.10.2
tomli 2.0.1
tox 3.26.0
urllib3 1.26.12
virtualenv 20.17.0
wheel 0.38.4
zipp 3.11.0
```

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

tests/test_integration.py、特に test_noquiet_installed_packages と test_regular_run_native_toxenv から始め、インストールプレフィックスに PYTHONPATH を設定した状態で、PEP517 wheel のインストール後に報告された pytest -ra の実行を再現します。失敗しているパッケージの順序、パッケージ数、exec_prefix のアサーションを、取得した出力と比較します。報告された非 root のパッケージングワークフローで、影響を受ける統合テストがパスすれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
python
領域
build-system, testing-qa
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。