python / python/cpython

Heap types implemented in C must have Py_TPFLAGS_HAVE_GC and implement tp_traverse

Open
#116,946 27 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

If a heap type does not have Py_TPFLAGS_HAVE_GC or doesn't not implement tp_traverse, the garbage collector may fail to collect the type and most of its instances at Python exit. The problem is worse when sub-interpreters are involved: if a sub-interpreter exits and does not clear all of its memory, the other interpreters will continue running with these leaks.

See issue gh-87138 for the rationale.

I wrote an article on the lock type which had this bug and caused a memory leak: https://vstinner.github.io/subinterpreter-leaks.html

Linked PRs
  • gh-116947
  • gh-138290
  • gh-138327
  • gh-138328
  • gh-138331
  • gh-138333
  • gh-138427
  • gh-138428
  • gh-138334
  • gh-138340
  • gh-138341
  • gh-138582
  • gh-138807
  • gh-139073
  • gh-139474
  • gh-141268
  • gh-141448
  • gh-152310
  • gh-156034

The following PRs were directly closed because the types are immutable and empty:

  • gh-138329
  • gh-138330
  • gh-138332

The following PRs were wrongly merged so we reverted them:

  • gh-138266
  • gh-138320
  • gh-138322
  • gh-138288
  • gh-138321
  • gh-138323
  • gh-138289
  • gh-138324
  • gh-138326

PRs for reverting the above commits:

  • gh-138338
  • gh-138337

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with issue gh-87138 and the linked article about the lock type, then inventory the C heap types covered by the linked PRs. Done means applicable heap types have Py_TPFLAGS_HAVE_GC and tp_traverse, with the described interpreter-exit and sub-interpreter leaks addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.