[BUG] Caching by Python version doesn't take alternate interpreters into account

setuptools version

setuptools==59.2.0

Python version

Python 3.7 & Python 3.8 for PyPy conflict

OS

All

Additional environment information

This is currently happening with PyPy 3.7 and PyPy 3.8 with CPython 3.7 and CPython 3.8, though it could happen with any conflicting pair of conflicting implementations, I would expect.

Description

Going to quote @mayeut for now, I can update later after more investigation. This is a PEP 517 build.

CIBW_BUILD=pp37-manylinux_x86_64 cibuildwheel --platform=linux succeeds (the wheel is copied back in my case).
CIBW_BUILD=*37-manylinux_x86_64 cibuildwheel --platform=linux fails.
This is true for both pip & build frontends.

The issue comes down to a lack of isolation between builds.
Adding before-build = "rm -rf {project}/build" solves the issue.
If using the pip frontend, before-build = "python -m pip install -U pip==21.2.4" also does the trick.
My best guess is that due to caching, cmake is trying to use the ninja it found during the cp37 build while building for pp37. That ninja is not present anymore after a the cp37 build ends.

We might want to report this to setuptools, it's probably not a good idea to cache builds for different interpreters (they already have python version segregation - 3.7, 3.8, ...-, it should probably go further than that).

I have not yet investigated the macOS failure but I can hazard that something similar is happening.

See https://github.com/pybind/cmake_example/pull/66

Expected behavior

Building a project with CPython 3.7 should not cause PyPy 3.7 to break. Ironically, the package here (ninja) is the same wheel on PyPy and CPython. (ninja-1.10.2.3-py2.py3-none-manylinux_2_5_x86_64.manylinux1_x86_64.whl)

How to Reproduce

Grab the cmake_example project from pybind11. Build it with CPython 3.7 then PyPy 3.7 without deleting the build folder in-between. The ninja command will not be found on the PyPy build. You can do it in either order.

Output

$ docker run --rm -it quay.io/pypa/manylinux2014_x86_64:latest
# git clone https://github.com/pybind/cmake_example --recurse-submodules
# python3.7 -m pip wheel ./cmake_example  # succeeds
# pypy3.7 -m pip wheel ./cmake_example
...
  CMake Error at CMakeLists.txt:2 (project):
    Running

     '/tmp/pip-build-env-b0erg7h_/overlay/bin/ninja' '--version'

    failed with:

     No such file or directory
...
# rm -rf cmake_example/build
# pypy3.7 -m pip wheel ./cmake_example  # succeeds
# python3.7 -m pip wheel ./cmake_example
...
  CMake Error at CMakeLists.txt:2 (project):
    Running

     '/tmp/pip-build-env-b0erg7h_/overlay/bin/ninja' '--version'

    failed with:

     No such file or directory
...

Code of Conduct

• I agree to follow the PSF Code of Conduct

Start by reproducing the failure in the pybind11 cmake_example project with the two pip wheel commands shown, building with CPython 3.7 and PyPy 3.7 without removing build. Inspect how setuptools caches build environments across Python implementations. Done means consecutive CPython and PyPy builds succeed without manual cleanup or interpreter-specific workarounds.