import_importlib leaks module attr sets
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
Our CI (https://github.com/facebookincubator/cinder/actions/runs/29382592631/job/87249270535) was failing on test_pickle and test_pickletool with:
PicklingError: Can't pickle
<class 'importlib._bootstrap.BuiltinImporter'>:
it's not the same object as
importlib._bootstrap.BuiltinImporter
I believe this is happening because we run multiple test modules in the same interpreter, so the following sequence happens:
test_importlib.util.import_importlib()imports a source copy ofimportlibwhile blocking_frozen_importlib. During this import,importlib._bootstrap._setup()initializes import metadata on existing built-in modules. Ifbuiltins.__loader__orbuiltins.__spec__was originally absent, the source copy installs its ownBuiltinImporter.- Although
import_fresh_module()restoressys.modules, it does not restore attributes mutated on existing module objects. Consequently,builtins.__loader__continues to reference the temporary sourceBuiltinImporter, whileimportlib._bootstrap.BuiltinImporterresolves to the restored frozen class. - Pickle serializes classes by module and qualified name and verifies that the resolved global is the same object. The two
BuiltinImporterclass objects therefore cause the identity check to fail.
To fix this we need to snapshot __loader__ and __spec__ before importing importlib and then restore them to their original values after.
CPython versions tested on:
CPython main branch
Operating systems tested on:
macOS
Linked PRs
- gh-154276
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with test_importlib.util.import_importlib(), then reproduce the failure in test_pickle and test_pickletool. Trace how the temporary importlib import changes builtins.loader and builtins.spec; done means the original values are restored and both tests pass without the PicklingError. A linked PR, gh-154276, indicates work is already underway.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 35/100