Add a private API to defer automatic garbage collection
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
I would like to propose a private API that allows CPython internals to defer automatic garbage collection on the current thread without changing the interpreter-visible GC state.
Some CPython internals build large object graphs in a burst, where a collection triggered halfway through cannot reclaim anything from that operation and only traverses a half-built graph. Using PyGC_Disable() for this changes interpreter-wide state. Saving and restoring that state is racy in the free-threaded build, and it can also be observed or changed by user code in the GIL build.
What we want is something approximately like:
_PyGC_DeferAutomaticCollection(tstate);
/* Build a large object graph. */
_PyGC_ResumeAutomaticCollection(tstate);
This would be a private CPython API. It should be per-thread and nestable, keep allocation counters running, leave gc.isenabled() unchanged, and defer only automatic collection. Leaving the outermost scope would schedule a collection if one became due.
I think there are many possible cases in the stdlib and core that can benefit from this, and we currently do not have a safe tool for them. This can be implemented with a small nesting counter in _PyThreadStateImpl, checked by the allocation paths and _Py_RunGC(). This would allow these optimizations without changing user-visible GC state or introducing cross-thread races.
Linked PRs
- gh-154188
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing _PyThreadStateImpl, the allocation paths, and _Py_RunGC(), which the proposal identifies as the relevant entry points. Compare the behavior with PyGC_Disable() and the linked PR gh-154188. Done should satisfy per-thread nesting, preserve allocation counters and gc.isenabled(), defer only automatic collection, and schedule a due collection when the outermost scope resumes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100