Improving GC collections: dynamic thresholds, single generation gc and time barriers
还没有人认领这个 Issue。
评估
- 难度
- 5/5
- 预计耗时
- 一周以上
- 新手友好度
- 20/100
- Issue 类型
- 功能
- 描述清晰度
- 需要澄清
- 活跃度
- 停滞
- 技术栈
- python
- 领域
- backend, performance
调研方向
首先查看此 issue 中的基准测试观察结果以及链接的 PR gh-100958。提议的工作涉及将当前的分代策略与单代和动态阈值进行比较；完成这项工作需要确定一种有性能结果支持的方法，但这里没有指定具体的文件或测试。
由索引模型根据 Issue 内容生成。
描述
In the pursuit of trying to optimize GC runs it has been observed that the weak generational hypothesis may not apply that well to Python. This is because, according to this argument, in the presence of a mixel cycle GC + refcount GC strategy, young objects are mostly cleaned up by reference count, not by the GC. Is important as well that there is no segregation between the GC strategies and that the cycle GC needs to deal with objects that according to this will be mainly cleaned up by reference count alone.
This questions the utility of segregating GC by generations and indeed there is some evidence of this. I have been benchmarking the percentage of success of different generations in some programs (such as blach and mypy and a bunch of HTTP servers) and the success rate of the lower generations is generally small. Here is an example of running black over all the standard library:
Statistics for generation 0
| Category | Value |
|---|---|
| count | 157917.000000 |
| mean | 1.192775 |
| std | 3.560391 |
| min | 0.000000 |
| 25% | 0.000000 |
| 50% | 0.000000 |
| 75% | 0.480192 |
| max | 86.407768 |
Statistics for generation 1
| Category | Value |
|---|---|
| count | 14346.000000 |
| mean | 2.670852 |
| std | 11.642815 |
| min | 0.000000 |
| 25% | 0.000000 |
| 50% | 0.000000 |
| 75% | 0.388794 |
| max | 97.406097 |
Statistics for generation 2
| Category | Value |
|---|---|
| count | 1280.000000 |
| mean | 45.698135 |
| std | 27.066735 |
| min | 0.000000 |
| 25% | 31.965862 |
| 50% | 54.618008 |
| 75% | 67.038842 |
| max | 90.592705 |
I am currently investigating if having a single generation with a dynamic threshold that is similar to the strategy that we use currently for the last generation would be generally better to get better performance.
What do you think?
Linked PRs
- gh-100958
- 主要语言
- Python
- 星标
- 77.2k
- 派生
- 36k
- 平均合并
- 1 天 9 小时
- 30 天内合并 PR
- 558
贡献指南
从这里开始
- 先读完整个 Issue，再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库，在一个分支上完成修改。
- 提交 Pull Request，并在描述里引用这个 Issue 编号。
python/cpython 的其他 Issue
-
docs pending
难度 2/5 1-3 小时 新手友好度 78/100
-
stdlib type-feature
难度 2/5 1-3 小时 新手友好度 78/100
-
stdlib type-feature
难度 2/5 1-3 小时 新手友好度 72/100
-
build type-bug
难度 2/5 1-3 小时 新手友好度 76/100
-
stdlib topic-email type-feature
难度 2/5 1-3 小时 新手友好度 70/100
相似的 Issue
-
难度 1/5 1 小时以内 新手友好度 90/100
-
bug
难度 2/5 1-3 小时 新手友好度 86/100
zostera/django-bootstrap4#894 ·
-
难度 2/5 1-3 小时 新手友好度 78/100
use-agent-os/agent-os#3276 ·
-
难度 2/5 1-3 小时 新手友好度 88/100
zephyrproject-rtos/zephyr#119726 ·
-
area/auth bug comp/agent P3 platform/discord type/security
难度 2/5 1-3 小时 新手友好度 88/100
NousResearch/hermes-agent#117848 ·