[Python] MacOS wheel builds fail after successfully building and installing CIRCTPythonModules.
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 2.2k
- Forks
- 524
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 46
Description
This appears to have been failing since we updated LLVM to integrate the upstream MLIR switch to nanobind for the core dialect [here](https://github.com/llvm/circt/commit/387291f6444ace38a132599a158db390bed3b92a).
I'm planning to remove this for now, since no one seems to care about MacOS wheels, and we have finite space in PyPI for CIRCT.
I do want to leave my thoughts, such as they are, in case anyone wants to try to pick this up.
There are a number of failed Upload Wheels jobs that I manually ran while testing, but take a look at one of the failures that was scheduled on main: https://github.com/llvm/circt/actions/runs/12632127552
In the failing MacOS job, it gets to the build_ext step, which is supposed to be a [no-op](https://github.com/llvm/circt/blob/4e478778cbe593af34695c2f454986a34b24c156/lib/Bindings/Python/setup.py#L133), but instead tries to run a half-baked C compile command, which fails: https://github.com/llvm/circt/actions/runs/12632127552/job/35195198998#step:6:7419
```
[...]
[100%] Built target install-CIRCTPythonModules
running build_ext
building 'circt._mlir_libs._mlir' extension
clang -bundle -undefined dynamic_lookup -g -arch x86_64 -o build/lib.macosx-10.9-x86_64-cpython-310/circt/_mlir_libs/_mlir.cpython-310-darwin.so
clang: error: no input files
error: command '/usr/bin/clang' failed with exit code 1
```
In a passing Linux job, the build_ext step really is a no-op: https://github.com/llvm/circt/actions/runs/12632127552/job/35195198360#step:6:7819
```
[...]
[100%] Built target install-CIRCTPythonModules
Detected static linked python. Faking a library for cmake.
running build_ext
running build_scripts
```
I tried a few things here. For one, I set a breakpoint in the NoopBuildExtension [method](https://github.com/llvm/circt/blob/4e478778cbe593af34695c2f454986a34b24c156/lib/Bindings/Python/setup.py#L133), and it never seems to get called. To sanity check myself, I did set a breakpoint in the CustomBuild, before and after [build_py](https://github.com/llvm/circt/blob/4e478778cbe593af34695c2f454986a34b24c156/lib/Bindings/Python/setup.py#L42), and the breakpoint before build_py was hit, but the failing build_ext step was run before the breakpoint after build_py was hit. So I think there is something wonky going on with setuptools/distutils, which has its own little command running framework. It feels like somehow that is conjuring a build_ext that isn't our overridden one.
I don't know why this would have changed recently. I don't know of anything upstream that would affect this, but maybe I'm missing something. I do know that we have an open ended setuptools dependency, which I don't like, so I tried pinning it to the oldest version of setuptools we can use, and that didn't fix this. I also know we recently switched to macos-13 runners, so perhaps that is related.
For now, I will plan to remove MacOS wheel builds in https://github.com/llvm/circt/pull/8051.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with lib/Bindings/Python/setup.py, especially NoopBuildExtension and CustomBuild around build_py, then compare the failing MacOS Upload Wheels job with the passing Linux job. Reproduce the build_ext step using the referenced GitHub Actions run and investigate the setuptools/distutils command dispatch. Done means the MacOS wheel workflow either reliably completes with the intended no-op or is removed as planned in PR 8051.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, python
- Domain
- build-system, ci-cd
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100