[FR] extensions: general way to provide library paths that are searched before `LDSHARED`'s paths
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 1.4k
- Avg merge
- 1d 1h
- Merged PRs (30d)
- 1
Description
What's the problem this feature will solve?
Currently, if sysconfig's LDSHARED contains a -L flag, that flag will appear before user-provided flags when building an extension. If the user provides an additional path to search (e.g. via LDFLAGS), that path will have lower priority than paths in LDSHARED since library paths are searched left to right. This makes it hard for a user to prefer their own library, even when an undesired version is being found via LDSHARED.
Describe the solution you'd like
Ideally, there would be a way to search user-provided paths first. But I'm honestly not sure what this would look like, implementation-wise. Since LDSHARED contains both the linker invocation and flags to the linker, any solution would seem to need to parse LDSHARED to insert -L flags early. That seems fragile. However, there is precedent in the sense that LDSHARED is already automatically rewritten with a user-provided CC: https://github.com/pypa/setuptools/blob/ba243756233d0afe944db6e02ddcb70064dcd22c/setuptools/_distutils/sysconfig.py#L331
Alternative Solutions
Users can currently pass LDSHARED to fully specify the linker command and the order of -L flags. This is currently the workaround I'm using. However, it's verbose (see below) and arguably error-prone, since it requires manually discovering the LDSHARED value and modifying it to insert your own paths first.
Additional context
Here's an example LDSHARED on my system, which contains a lot of paths a user might want to override (notably -L/usr/lib64):
> python -m sysconfig
...
LDSHARED = "/mnt/sw/nix/store/qgcgvdlgz6546x6mzjiknvrq218j9nx8-gcc-11.4.0/bin/gcc -pthread -shared -L/mnt/sw/nix/store/30zwhjaf2jh84r0bws1yz2dwgqhw23f0-bzip2-1.0.8/lib -L/mnt/sw/nix/store/q0d0y49rsz7291c00kr0560y2q6bix58-expat-2.6.2/lib -L/mnt/sw/nix/store/snfa89v7a38qmzwqsj7006awc2z77rf4-gdbm-1.23/lib -L/mnt/sw/nix/store/zdnba5ggf37q6cxx1rszxabi8m260cfh-gettext-0.22.4/lib -L/mnt/sw/nix/store/d9n53v93vxra9a0madzpkicmrxxgbd61-libffi-3.4.6/lib64 -L/mnt/sw/nix/store/6416cbjlqkw7l9q7naz81ha4mknw3pkr-libxcrypt-4.4.35/lib -L/mnt/sw/nix/store/hn8dq06sw3hyx5z21z1n2kf62d37n8sv-ncurses-6.4/lib -L/usr/lib64 -L/mnt/sw/nix/store/nfr8l146l2lfxhdphgmd63qb8awy07w7-readline-8.2/lib -L/mnt/sw/nix/store/i117x95p5jmjpqmkdp8iyvr6n4yhzn6i-sqlite-3.43.2/lib -L/mnt/sw/nix/store/0swn26vgd2lwfnindbxcgzbpz2ra2qmc-tcl-8.6.12/lib -L/mnt/sw/nix/store/wl8bvjd4iwsx496lbpy641ihpswldqry-tk-8.6.11/lib -L/mnt/sw/nix/store/rrkjvd4n6dg3y0wvr2r34zq6zfz7864a-util-linux-uuid-2.38.1/lib -L/mnt/sw/nix/store/pmzfq3af88pyl5ykcrxqjqa1k4639nnl-xz-5.4.6/lib -L/mnt/sw/nix/store/2i17p7mws5s8x6y68md7vyclaim4b7nk-zlib-1.3.1/lib"
...
Code of Conduct
- I agree to follow the PSF Code of Conduct
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in setuptools/_distutils/sysconfig.py around the existing LDSHARED rewrite for a user-provided CC, then trace how LDSHARED and LDFLAGS reach extension builds. Define a way for user-provided library paths to precede LDSHARED paths without requiring a complete LDSHARED override; done means the ordering is supported without fragile manual command parsing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100