[BUG] setuptools.Distribution may be leaking references

setuptools version

67.8.0

Python version

3.11.3

OS

macOS

Description

Some background info: When adding the new PEG parser to CPython back in 3.9, we also added some tests that test the new parser generator. In order to do so, the tests build an extension module with a lot of different grammar, use that to parse a piece of code, and then compare the resulting syntax trees with the expected one.

Up until 3.11, we were using distutils to build the extension module. After the removal of distutils in 3.12, we recently migrated into using setuptools in python/cpython#104798. Since that PR was merged, some buildbots fail due to this test cause of some refleaks (example). Me and @pablogsal were able to bisect that down to the call to setuptools.Distribution in setup.py.

We're still not sure whether that's a true refleak in setuptools or some caching that setuptools does, which leads to the CPython test suite to fail, so I'm opening this issue to hopefully start a discussion on whether CPython needs to change something or setuptools.

I'm also including below a little script that reproduces this without the need of the CPython test suite or anything else.

Expected behavior

No refleaks.

How to Reproduce

Run the following script:

import sys

from setuptools import Extension, Distribution

int_pool = {value: value for value in range(-1000, 1000)}
def get_pooled_int(value):
    return int_pool.setdefault(value, value)


def build():
    with open("hello.c", "w") as f:
        f.write("int spam() {\n")
        f.write("    return 1;\n")
        f.write("}\n")
    extension = Extension(
        "something",
        sources=["hello.c"],
    )
    Distribution({"name": "something", "ext_modules": [extension]})

def main(run_build_between=False):
    build()
    build()
    build()

    getallocatedblocks = sys.getallocatedblocks
    gettotalrefcount = sys.gettotalrefcount
    getunicodeinternedsize = sys.getunicodeinternedsize

    interned_before = getunicodeinternedsize()
    alloc_before = getallocatedblocks() - interned_before
    rc_before = gettotalrefcount() - interned_before * 2

    if run_build_between:
        build()

    interned_after = getunicodeinternedsize()
    alloc_after = getallocatedblocks() - interned_after
    rc_after = gettotalrefcount() - interned_after * 2
    rc_delta = get_pooled_int(rc_after - rc_before)
    alloc_delta = get_pooled_int(alloc_after - alloc_before)
    print(f"{run_build_between=}: {rc_delta=} {alloc_delta=}")


if __name__ == "__main__":
    main(run_build_between=False)
    main(run_build_between=True)

Output

run_build_between=False: rc_delta=3 alloc_delta=3
run_build_between=True: rc_delta=238 alloc_delta=180

Start with the supplied reproducer and the setuptools.Distribution call in CPython's Tools/peg_generator/pegen/build.py setup.py path. Compare refcounts with and without the intervening build, then determine whether Distribution retains references or caching is involved; done means the excess references are explained and the reported refleak is resolved or clearly assigned to the correct project.