Empty `torch_mlir._mlir_libs._jit_ir_importer` lib causes package build problem on macOS
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 736
- Avg merge
- 5d 22h
- Merged PRs (30d)
- 15
Description
There's something strange about how `torch_mlir._mlir_libs._jit_ir_importer` is defined in the build system.
When building Python packages, `jit_ir_importer` is declared as an extension: https://github.com/llvm/torch-mlir/blob/9a6fe58a027d701eff6799e86a65535a8c2f3708/setup.py#L238-L242
It's declared without a build recipe or source files, so it's built as an empty library. To replicate that, you can run `python setup.py build_ext` (which is a part of package build process typically invoked by `python setup.py bdist_wheel`). If you run it with `-vv` switch, you'll see the build command that builds the library (this is on Linux):
```
> python setup.py build_ext -vv
running build_ext
building 'torch_mlir._mlir_libs._jit_ir_importer' extension
x86_64-linux-gnu-gcc -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -g -fwrapv -O2 -L/usr/lib/x86_64-linux-gnu -o setup_build/lib.linux-x86_64-cpython-311/torch_mlir/_mlir_libs/_jit_ir_importer.cpython-311-x86_64-linux-gnu.so
```
Look closely at gcc invocation. It has output library specified, but doesn't have any inputs specified!!! So the result is empty lib with just boilerplate code. And gcc does not complain, just silently produces empty lib.
But when building on macOS, clang is used instead of gcc and it does complain about no inputs provided:
```
> python setup.py build_ext -vv
running build_ext
clang -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX14.sdk -L/opt/homebrew/opt/libomp/lib -I/opt/homebrew/opt/libomp/include -o setup_build/lib.macosx-14.0-arm64-cpython-312/torch_mlir/_mlir_libs/_jit_ir_importer.cpython-312-darwin.so
clang: error: no input files
error: command '/usr/bin/clang' failed with exit code 1
```
And yes, clang is absolutely right, no inputs are provided.
But it's not the end of the story on macOS. With some Python distros it's reproducible and with some not. As it appears, the reason is this patch https://github.com/pypa/setuptools/commit/30b7331b07fbc404959cb37ac311afdfb90813be which already propagated to 3.12 `setuptools`, but looks like not to 3.11 yet (not sure how this really works). Before the patch `setuptools` were skipping the build of `torch_mlir._mlir_libs._jit_ir_importer` as the library has no source, but after the patch if the library is missing, it's being built even if it doesn't have sources. And that breaks macOS build.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with setup.py lines 238-242 and reproduce the issue using python setup.py build_ext -vv, then check the package build path invoked by python setup.py bdist_wheel on macOS. Compare the extension declaration and setuptools behavior across the affected Python versions. Done means the macOS package build completes without the empty _jit_ir_importer compilation failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100