ABC causing massive memory swell

首先，在受影响的 Python 版本上运行所提供的复现程序，并测量 issubclass 循环期间的内存使用情况。阅读涉及的 ABCMeta 和 issubclass 行为，然后将结果与 issues #92810 和 pydantic #3829 进行比较。当一个经过协商并测试的方案解决缓存增长和子类树遍历问题，同时不使 ABC 注册行为发生回归时，即视为完成。

Bug report

from abc import ABC, ABCMeta
from datetime import datetime


abcclasses = set()

normalclasses = set()

for i in range(10000):
    abcclasses.add(ABCMeta("abc_"+str(i), (ABC, ), {}))
    normalclasses.add(type("normal_"+str(i), (object,), {}))


if __name__ == '__main__':

    starttime = datetime.now()
    import os, psutil
    process = psutil.Process(os.getpid())
    mb = 1024 * 1024
    mem = last = process.memory_info().rss
    print(f'{i + 1:>4d} {mem / mb:8.2f}MB {(mem - last) / mb:+8.2f}MB | {"━" * int(mem / 8_000_000)}')
    for item in normalclasses:
        issubclass(item, ABC)

    mem = process.memory_info().rss
    print(f'{i + 1:>4d} {mem / mb:8.2f}MB {(mem - last) / mb:+8.2f}MB | {"━" * int(mem / 8_000_000)}')
    print(f"This took {datetime.now()-starttime}")

Your environment

Running this takes several minutes and consumes over 20 gbs of memory.

I've checked it against python 3.8-3.10. The issue is just that ABC caches every single issubclass evaluation combined with it searching its entire subclass tree recursively causes it to grind to a halt if you have any significant number of subclass 10k sub classes of ABC and 10k normal classes being compared to the base ABC climbs to over 20 GBs. I am not really sure it is necessary to check the entire subclass tree. I presume it is because of concerns about register calls happening low in the class tree.

It would likely be far more efficient to chase upward the calls to register and place those in the parent classes then to search the entire class hierarchy. This can get really aggressive performance issues if you have any multiple inheritance structures going on as well as those will get checked multiple times.

See these issues.
https://github.com/python/cpython/issues/92810
https://github.com/samuelcolvin/pydantic/issues/3829