mesonbuild / mesonbuild/meson-python
pytest assertion rewriting does not seem to work with meson editable install
- Dominant language
- Python
- Stars
- 180
- Forks
- 93
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 18
Description
It could well be partially linked to some of the discussion in https://github.com/mesonbuild/meson-python/issues/642 but I thought a separate issue may be worth it.
### Description of the issue
The pytest assertion rewriting that shows additional debug information when an assert fails does not show up when using meson editable install. It does work fine when using normal (non-editable) meson install. Although not crucial I would argue that this can be actually quite useful in CI logs.
It could also well be a Pytest issue but I thought I had to start somewhere and chose meson-python.
More background about Pytest assertion rewriting, see [Pytest doc](https://docs.pytest.org/en/stable/how-to/assert.html#assertion-introspection-details) that links to a somewhat old [blog post](http://pybites.blogspot.com/2011/07/behind-scenes-of-pytests-new-assertion.html) but I am guessing is probably still relevant?
This was originally seen in scikit-learn (see https://github.com/scikit-learn/scikit-learn/issues/29253#issuecomment-2173549242 if you are really curious where it would have actually been quite useful to see intermediary values in the `AssertionError` in https://github.com/scikit-learn/scikit-learn/issues/29253#issuecomment-2167610891). I put together a reproducer below with numpy hoping this makes it a bit easier to look at. I guess a smaller reproducer could be put together if deemed useful, but it would take a bit of time (I tried with https://github.com/dnicolodi/python-siphash24 and I am not able to reproduce the issue).
### Get the reproducer from my numpy branch
```
git clone https://github.com/lesteve/numpy
cd numpy
git checkout meson-editable-pytest-assertion-rewrite
```
I added a simple test in `numpy/tests/test_mine.py`
```
import numpy as np
def test():
arr = np.arange(100)
assert arr.sum() < 100
```
When running with pytest you pytest assertion rewriting shows the value or `arr.sum()` (see below).
### Normal (non-editable) install
```
pip install --verbose --no-build-isolation .
```
Note the assertion rewriting i.e. you get the value of `arr.sum()`
Partial output:
```
E assert np.int64(4950) < 100
E + where np.int64(4950) = ()
```
Full output
```
❯ pytest --pyargs numpy.tests.test_mine
========================== test session starts ===========================
platform linux -- Python 3.11.7, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/lesteve
plugins: hypothesis-6.103.2
collected 1 item
micromamba/envs/numpy-dev/lib/python3.11/site-packages/numpy/tests/test_mine.py F [100%]
================================ FAILURES ================================
__________________________________ test __________________________________
def test():
arr = np.arange(100)
> assert arr.sum() < 100
E assert np.int64(4950) < 100
E + where np.int64(4950) = ()
E + where = array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,\n 17, 18, 19, 20, 21, 22, 23, 24, 25, ...72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84,\n 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99]).sum
micromamba/envs/numpy-dev/lib/python3.11/site-packages/numpy/tests/test_mine.py:6: AssertionError
======================== short test summary info =========================
FAILED micromamba/envs/numpy-dev/lib/python3.11/site-packages/numpy/tests/test_mine.py::test - assert np.int64(4950) < 100
=========================== 1 failed in 0.82s ============================
```
### Editable install
```
pip install --verbose --no-build-isolation --editable . --config-settings editable-verbose=true
```
You don't get the value of `arr.sum()`
```
pytest --pyargs numpy.tests.test_mine
========================== test session starts ===========================
platform linux -- Python 3.11.7, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/lesteve
plugins: hypothesis-6.103.2
collecting ... + /home/lesteve/micromamba/envs/numpy-dev/bin/ninja
[1/1] Generating numpy/generate-version with a custom command
Saving version to numpy/version.py
collected 1 item
dev/numpy/numpy/tests/test_mine.py F [100%]
================================ FAILURES ================================
__________________________________ test __________________________________
def test():
arr = np.arange(100)
> assert arr.sum() < 100
E AssertionError
dev/numpy/numpy/tests/test_mine.py:6: AssertionError
======================== short test summary info =========================
FAILED dev/numpy/numpy/tests/test_mine.py::test - AssertionError
=========================== 1 failed in 1.02s ============================
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the reproducer in numpy/tests/test_mine.py and compare pytest output after the documented normal and editable install commands. Trace how the editable installation affects pytest assertion rewriting, using the linked Meson issue for context. Done means the editable-install run reports the evaluated assertion values, matching the normal-install behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100