pypa / pypa/setuptools

[build_ext] bulding shared libraries: sysconfig._CONFIG_VARS vs distutils.sysconfig._config_vars

Open
#571 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Needs Triage
Dominant language
Python
Stars
2.9k
Forks
1.4k
Avg merge
1d 1h
Merged PRs (30d)
1

Description

Hello,

I was browsing the setuptools sources, and I discovered that it has (experimental?) support for building native shared libraries (.dylib, .dll, etc.) to be linked with Python extension modules.
I'm referring to setuptools.extension.Library class and the setup_shlib_compiler method of build_ext command.

I also noticed there was a tests/shlib_test (which somehow was removed with commit https://github.com/pypa/setuptools/commit/654c26f78a3031c34b81f338d3b6d1a3a87c34da). I only had to modify the "shlib_test/setup.py" script a bit to make it run with the current setuptools (21.0):

-from setuptools import setup, Extension, Library
+from setuptools import setup, Extension
+from setuptools.extension import Library

I encountered an issue in the setup_shlib_compiler method of build_ext command, where the compiler is configured for building shared libraries. When the platform is OSX ("darwin"), the global _CONFIG_VARS are temporarily modified (the flags -dynamiclib -undefined dynamic_lookup are required for .dylib); then the distutils.sysconfig.customize_compiler function is called in order to set these flags.
The problem is the distutils' customize_compiler function is reading from a different dictionary than then one setuptools is modifying, namely distutils.sysconfig._config_vars; therefore, the default values are used instead of the temporarily modified ones, and a -bundle (which can't be linked at compile time) instead of a -dynamiclib gets created.

The issue was introduced with this old commit https://github.com/pypa/setuptools/commit/c04abca6 which replaced all uses of distutils.sysconfig in setuptools, including build_ext module, with equivalent imports from the new sysconfig module, available since Python >= 2.7/3.2.

The issue can be fixed by importing _config_vars from distutils.sysconfig instead of the top-level sysconfig. Then the customize_compiler function can use the temporarily overridden variables as originally intended.

I hope this makes sense... :)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the build_ext.setup_shlib_compiler method and compare its sysconfig._CONFIG_VARS import with distutils.sysconfig._config_vars and customize_compiler. Restore or adapt the tests/shlib_test setup.py entry point described in the issue, then verify that the OSX shared-library build uses the required dynamic-library flags rather than bundle flags.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.