python / python/cpython

Improvements Related to Namespace Package Portions for PathEntryFinders

Open
#92,896 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core topic-importlib type-feature
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

tl;dr there are some oddities we should consider addressing with namespace package portions coming from path entry finders. It probably isn't worth bothering though (except for the docs fixes).


PEP 420 added (implicit) namespace packages. At that time, and since then, namespace packages (and portions) have gotten special-casing throughout the import machinery. Generally, we treat them like normal modules (or specs) even though certain characteristics don't apply. This isn't a problem generally because namespace packages aren't that common (and because, I expect, only builtin importers ever deal with them). However, there are a few things that it might be worth addressing.

PEP 451 introduced ModuleSpec, etc. PathEntryFinder.find_spec() as the replacement for PathEntryFinder.find_loader(), with some notable differences:

  • find_loader() returns (loader, portion)
    • the interpretation of this value is clearly documented
    • the finder can't provide any other possible information
    • if the returned loader is not None then the portion is ignored (not even a warning is set)
  • find_spec() returns a ModuleSpec with loader set to None and submodule_search_locations set to the portion
    • the interpretation of this value for portions is not clearly documented anywhere
    • the path entry finder may set addition info on the spec (e.g. loader_state) but it is all ignored (not even a warning if set)

In both cases, the language reference is not clear about how to identify namespace package portions from a path entry finder.

What to Do?

  • update the PathEntryFinder.find_spec() docs entry
  • (a weak maybe) add the details to the language reference
  • (maybe) be more explicit when returning a namespace package portion
  • (maybe) strictly disallow (or warn) for extraneous information tied to a portion
explicitly identify namespace package portions
  1. do nothing
  2. use a dedicated singleton instead of None
  3. use a dedicated ModuleSpec subclass (e.g. NamespacePackagePortionSpec)
  4. use a dedicated ModuleSpec-compatible class (e.g. NamespacePackagePortionSpec)
explicitly deal with unexpectedly set data

(This is basically just if spec.origin is set.)

  1. do nothing
  2. emit a warning
  3. fail
explicitly deal with spec.loader_state
  1. do nothing
  2. emit a warning
  3. fail
  4. preserve the info on the spec of the combined namespace package

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the PathEntryFinder.find_spec() documentation entry and the language-reference namespace-packages section linked in the issue. Read the PEP 420 and PEP 451 context, then review the unresolved alternatives for identifying portions and handling extra spec data. Done means the project’s chosen behavior is documented clearly, with any related behavior changes agreed before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation, tooling
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.