3.11 no longer builds with a configured empty prefix

Open
#101,668 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
c, python
Domain
build-system

Research direction

Reproduce the failure with ./configure --prefix= followed by make, then trace the prefix values through Modules/getpath.py, Modules/getpath.c, Python/initconfig.c, and Python/sysmodule.c. Compare the C-to-Python conversion with the earlier behavior referenced in the issue. Done means CPython builds successfully with an empty prefix and the affected prefix attributes remain usable.

Written by the indexing model from the issue text.

Description

build type-bug

Bug report

In 3.11 is not possible to build by running ./configure with an empty prefix:

./configure --prefix=

After such configuration, running make fails with something similar to the following:

./_bootstrap_python ./Programs/_freeze_module.py abc ./Lib/abc.py Python/frozen_modules/abc.h
Fatal Python error: init_import_site: Failed to import the site module
Python runtime state: initialized
Traceback (most recent call last):
  File "/home/etanol/Python-3.11.1/Lib/site.py", line 79, in <module>
    PREFIXES = [sys.prefix, sys.exec_prefix]
                ^^^^^^^^^^
AttributeError: module 'sys' has no attribute 'prefix'
Makefile:1200: recipe for target 'Python/frozen_modules/abc.h' failed

Possible explanation

The sys module takes attributes like sys.exec and sys.exec_prefix from the PyConfig structure. However, if any of those values is NULL, the attribute does not seem to be defined.

During path calculation, the result of evaluating (the frozen version of) getpath.py is copied back to PyConfig. But the prefix and similar fields are treated by converting None to NULL (from the C macro expansion).

The prefix indicated to ./configure is passed on when compiling getpath.c. But empty strings are converted to None. If the evaluation of (frozen) getpath.py assigns None to the prefix global variable (and similar) what gets transferred back to getpath.c ends up becoming NULL in PyConfig.

I think an empty prefix worked in 3.10, and earlier, because the transition from C compiler macro to Python value preserved empty strings. Empty C strings were converted to empty Python strings; but 3.11 converts empty C strings to None.

Considerations

Using dummy values for ./configure --prefix and then controlling installation target with make DESTDIR=... is a fairly common practice when creating binary packages. It's very convenient as a way to build software inside sandboxed environments.

However, it is certainly arguable that allowing for an empty prefix may not be of interest (it's just as common to run ./configure --prefix=/).

It is also beyond my understanding to determine if the fix should be about preserving empty strings in the C to Python conversion or perhaps the logic in getpath.py should be modified with a last resort check to avoid setting prefix (and similar) to None.

Your environment

  • CPython versions tested on: 3.11.1
  • Operating system and architecture: Ubuntu 18.04 x86-64
Dominant language
Python
Stars
77.2k
Forks
36k
Avg merge
1d 9h
Merged PRs (30d)
558

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from python/cpython

All issues in python/cpython

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.