llvm / llvm/circt

[Python] MacOS wheel builds fail after successfully building and installing CIRCTPythonModules.

Open
#8,054 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Python
Dominant language
C++
Stars
2.2k
Forks
524
Avg merge
3d 2h
Merged PRs (30d)
46

Description

This appears to have been failing since we updated LLVM to integrate the upstream MLIR switch to nanobind for the core dialect [here](https://github.com/llvm/circt/commit/387291f6444ace38a132599a158db390bed3b92a).

I'm planning to remove this for now, since no one seems to care about MacOS wheels, and we have finite space in PyPI for CIRCT.

I do want to leave my thoughts, such as they are, in case anyone wants to try to pick this up.

There are a number of failed Upload Wheels jobs that I manually ran while testing, but take a look at one of the failures that was scheduled on main: https://github.com/llvm/circt/actions/runs/12632127552

In the failing MacOS job, it gets to the build_ext step, which is supposed to be a [no-op](https://github.com/llvm/circt/blob/4e478778cbe593af34695c2f454986a34b24c156/lib/Bindings/Python/setup.py#L133), but instead tries to run a half-baked C compile command, which fails: https://github.com/llvm/circt/actions/runs/12632127552/job/35195198998#step:6:7419

```
[...]
[100%] Built target install-CIRCTPythonModules
running build_ext
building 'circt._mlir_libs._mlir' extension
clang -bundle -undefined dynamic_lookup -g -arch x86_64 -o build/lib.macosx-10.9-x86_64-cpython-310/circt/_mlir_libs/_mlir.cpython-310-darwin.so
clang: error: no input files
error: command '/usr/bin/clang' failed with exit code 1
```

In a passing Linux job, the build_ext step really is a no-op: https://github.com/llvm/circt/actions/runs/12632127552/job/35195198360#step:6:7819

```
[...]
[100%] Built target install-CIRCTPythonModules
Detected static linked python. Faking a library for cmake.
running build_ext
running build_scripts
```

I tried a few things here. For one, I set a breakpoint in the NoopBuildExtension [method](https://github.com/llvm/circt/blob/4e478778cbe593af34695c2f454986a34b24c156/lib/Bindings/Python/setup.py#L133), and it never seems to get called. To sanity check myself, I did set a breakpoint in the CustomBuild, before and after [build_py](https://github.com/llvm/circt/blob/4e478778cbe593af34695c2f454986a34b24c156/lib/Bindings/Python/setup.py#L42), and the breakpoint before build_py was hit, but the failing build_ext step was run before the breakpoint after build_py was hit. So I think there is something wonky going on with setuptools/distutils, which has its own little command running framework. It feels like somehow that is conjuring a build_ext that isn't our overridden one.

I don't know why this would have changed recently. I don't know of anything upstream that would affect this, but maybe I'm missing something. I do know that we have an open ended setuptools dependency, which I don't like, so I tried pinning it to the oldest version of setuptools we can use, and that didn't fix this. I also know we recently switched to macos-13 runners, so perhaps that is related.

For now, I will plan to remove MacOS wheel builds in https://github.com/llvm/circt/pull/8051.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with lib/Bindings/Python/setup.py, especially NoopBuildExtension and CustomBuild around build_py, then compare the failing MacOS Upload Wheels job with the passing Linux job. Reproduce the build_ext step using the referenced GitHub Actions run and investigate the setuptools/distutils command dispatch. Done means the MacOS wheel workflow either reliably completes with the intended no-op or is removed as planned in PR 8051.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, python
Domain
build-system, ci-cd
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.