python / python/cpython

Improving GC collections: dynamic thresholds, single generation gc and time barriers

未关闭
#100,403 29 条评论 3 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

interpreter-core
主要语言
Python
星标
77.2k
派生
36k
平均合并
1 天 9 小时
30 天内合并 PR
558

描述

In the pursuit of trying to optimize GC runs it has been observed that the weak generational hypothesis may not apply that well to Python. This is because, according to this argument, in the presence of a mixel cycle GC + refcount GC strategy, young objects are mostly cleaned up by reference count, not by the GC. Is important as well that there is no segregation between the GC strategies and that the cycle GC needs to deal with objects that according to this will be mainly cleaned up by reference count alone.

This questions the utility of segregating GC by generations and indeed there is some evidence of this. I have been benchmarking the percentage of success of different generations in some programs (such as blach and mypy and a bunch of HTTP servers) and the success rate of the lower generations is generally small. Here is an example of running black over all the standard library:

Statistics for generation 0

Category Value
count 157917.000000
mean 1.192775
std 3.560391
min 0.000000
25% 0.000000
50% 0.000000
75% 0.480192
max 86.407768

Statistics for generation 1

Category Value
count 14346.000000
mean 2.670852
std 11.642815
min 0.000000
25% 0.000000
50% 0.000000
75% 0.388794
max 97.406097

Statistics for generation 2

Category Value
count 1280.000000
mean 45.698135
std 27.066735
min 0.000000
25% 31.965862
50% 54.618008
75% 67.038842
max 90.592705

I am currently investigating if having a single generation with a dynamic threshold that is similar to the strategy that we use currently for the last generation would be generally better to get better performance.

What do you think?

Linked PRs
  • gh-100958

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

首先查看此 issue 中的基准测试观察结果以及链接的 PR gh-100958。提议的工作涉及将当前的分代策略与单代和动态阈值进行比较;完成这项工作需要确定一种有性能结果支持的方法,但这里没有指定具体的文件或测试。

由索引模型根据 Issue 内容生成。

评估

技术栈
python
领域
backend, performance
Issue 类型
功能
难度
5/5
预计耗时
一周以上
活跃度
停滞
描述清晰度
需要澄清
新手友好度
20/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。