python / python/cpython

Custom environments in subinterpreters

Open
#126,977 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

topic-subinterpreters type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

Proposal:

I wanted to explore the viability of having custom environments in subinterpreters. There are several use-cases that could be enabled by this feature.

So far, from informal discussion with others about this, there are a couple possible issues to take into consideration.

Issues

  1. Some of the immortal objects shared between subinterpreter may be environment-dependent (pointed out by @Yhg1s)
  2. Complications around dynamic loading, by having extension modules from different environments
    2.1) Symbol conflicts from their dependencies (pointed out by @Yhg1s)
    2.2) Since subinterpreters share the same process, when loading the same shared object, they get the same pointer (pointed out by @pablogsal)

Implementation

The main thing we need is a way to disable the site initialization, which could be a enable_site option in the interpreter config. This should disable the environment customizations, and result in a bare environment without anything extra sys.path.

However, to make the use of different environments more ergonomic, we could add an environment_path location pointing to a directory containing a pyvenv.cfg, which would perform the site initialization for that environment.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files or tests; start by reviewing subinterpreter interpreter configuration and site initialization. Evaluate the proposed enable_site and environment_path options against the listed immortal-object and dynamic-loading concerns. Done requires an agreed design that resolves those concerns and defines the implementation scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.