PennyLaneAI / PennyLaneAI/catalyst

Local Catalyst wheel build is broken on macOS

Open
#1,453 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
234
Forks
84
Avg merge
2d 15h
Merged PRs (30d)
66

Description

On my system (macOS 14 arm64), I can no longer build working wheels from a local repository. This is present on both v0.10 and v0.9. Besides changes in the Catalyst package, it could also be due to a change in build environment (OS or package updates).

The problem is down to the unique library ID of the custom_calls library as seen below (first line of a library is always itself):

otool -L PennyLane_Catalyst-0.9.0/catalyst/utils/libcustom_calls.so 
PennyLane_Catalyst-0.9.0/catalyst/utils/libcustom_calls.so (architecture x86_64):
        build/lib.macosx-10.9-universal2-cpython-311/catalyst/utils/libcustom_calls.so (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1800.101.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
PennyLane_Catalyst-0.9.0/catalyst/utils/libcustom_calls.so (architecture arm64):
        build/lib.macosx-10.9-universal2-cpython-311/catalyst/utils/libcustom_calls.so (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1800.101.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)

while the library is originally generated with the correct identifier (@rpath/libcustom_calls.so), when packaged into the wheel the identifier changes to build/lib.macosx-10.9-universal2-cpython-311/catalyst/utils/libcustom_calls.so which will cause a dynamic linker error when loading the custom calls library during execution.

I briefly tested to see if delocate would fix the wheel, but it doesn't. Wheels on the CI seem as of yet unaffected by this.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the local wheel build on macOS 14 arm64 and inspect catalyst/utils/libcustom_calls.so with otool -L. Compare the original @rpath/libcustom_calls.so identifier with the packaged wheel's build/... identifier, and check whether delocate changes the result. Done means locally built wheels retain a loadable library identifier and custom calls load during execution.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.