pybind / pybind/pybind11

[BUG]: Cmake cannot find Python include directories for Python 3.11 on MacOS

Open
#4,624 6 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

triage
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

ed466da571fbc1d711351eb818e4bf82adb99eca

Problem description

When compiling a project which uses Pybind11 (oxDNA, update_pybind branch), compilation fails when using Python 3.11 on Mac OS in a base Conda environment because it finds the incorrect include directory. Compilation works with with 3.11 on Linux, or on Mac if I create a fresh environment with 3.11 (or any other version) and compile there.

If set(PYBIND11_FINDPYTTHON True) is not included in our CMakeLists.txt, Cmake generates the following error:

CMake Error in oxpy/CMakeLists.txt:
  Imported target "pybind11::module" includes non-existent path

    "~/miniconda3/python.app/Contents/include/python3.11"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.

The actual include directory is ~/miniconda3/include/python3.11/

If, however, set(PYBIND11_FINDPYTTHON True) is included in our CMakeLists.txt as suggested in the documentation, no errors are emitted by Cmake, but the PYTHON_INCLUDE_DIRS variable is empty and the compilation fails with

In file included from ~/software/oxDNA/oxpy/pybind11/include/pybind11/detail/../attr.h:13:
~/software/oxDNA/oxpy/pybind11/include/pybind11/detail/common.h:266:10: fatal error: 'Python.h' file not found

So while this could also be a problem with Cmake or Conda, I suspect something is misconfigured in the Pybind11 CmakeList.txt for this specific case.

Reproducible example code

No response

Is this a regression? Put the last known working version here if it is.

This worked in Pybind11 2.2.4, but we have not tested intervening versions.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with pybind11 commit ed466da and Python 3.11 in a base Conda environment on macOS, then inspect the CMake Python discovery configuration referenced by the project CMakeLists.txt. Compare the incorrect python.app/Contents/include path and the actual ~/miniconda3/include/python3.11 path; done means both discovery modes expose the valid Python headers and the reported compilation failure is resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.