python / python/cpython

macOS linker warnings in macOS ventura

Open
#97,524 55 comments 3 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3.10 3.11 3.12 3.13 3.9 (EOL) OS-mac
Dominant language
Python
Stars
77.2k
Forks
36k
Avg merge
1d 9h
Merged PRs (30d)
558

Description

Turns out ld shipped with Xcode 14+ emits a warning when using -undefined dynamic_lookup.`

ld: warning: -undefined dynamic_lookup may not work with chained fixups

Some investigation reveals that in fact -undefined dynamic_lookup doesn't work when:

  • Link a shared library with the option
  • Link it with a program that uses the chained-fixup introduced from macOS 12 and iOS 15

This is because -undefined dynamic_lookup uses lazy-bindings and they won't be bound while dyld fixes-up by traversing chained-fixup info.

However, we build extension modules with as bundles (-bundle) and they are loaded only through
dlopen, so it's safe to use -undefined dynamic_lookup in theory. So the warning produced by ld64 is
likely to be a false-positive.

ld64 also provides the -bundle_loader <executable> option, which allows resolving symbols defined in the executable symbol table while linking. It behaves almost the same with -undefined dynamic_lookup, but it makes the following changes:

  • Require that unresolved symbols among input objects must be defined in the executable.
  • Lazy symbol binding will lookup only the symbol table of the bundle loader executable. (-undefined dynamic_lookup lookups all symbol tables as flat namespace)

See "New Features" subsection under "Linking" section for chained fixup
developer.apple.com/documentation/xcode-release-notes/xcode-13-release-notes for more information.

Also, check the same problem in ruby where most of this information is taken from.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by locating CPython's macOS extension-module linker configuration and the uses of -undefined dynamic_lookup; compare the current behavior with ld64's -bundle_loader option. Done means the Ventura/Xcode 14+ warning is addressed without breaking extension-module loading.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, python
Domain
build-system, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.