make marshal output not dependent on reference count
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 77.2k
- 派生
- 36k
- PR 合并指标
- PR 指标待抓取
描述
Bug report
Currently the marshal module will emit a previously-unseen object flagged as a potential reference from later objects, unless the object has a reference count of 1. See https://github.com/python/cpython/blob/b27b57c6e44a276c8a9843fd37d4cf65b2827d5c/Python/marshal.c#L305
This is an overly-conservative heuristic -- it's easy to construct cases where an object has a reference count >1 but is not actually referenced by any other object about to be marshaled, so FLAG_REF is set when it does not need to be.
This makes marshal output unstable depending on accidents of reference counting behavior in the code calling marshal.dumps.
I ran into this because the Cinder JIT is able to reduce unnecessary increfs, and that resulted in some importlib tests failing on comparison of marshal output at https://github.com/python/cpython/blob/b27b57c6e44a276c8a9843fd37d4cf65b2827d5c/Lib/test/test_importlib/test_abc.py#L870-L871 because under Cinder JIT the reference count of code_object in that method is 1.
This previously caused issues in distutils reproducibility, resulting in a partial fix that applies only to interned strings: https://github.com/python/cpython/pull/8226
It would be better if marshal would actually determine which objects have multiple parents in the DAG and deterministically use FLAG_REF or not based on that.
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
从 Python/marshal.c 中提到的引用计数启发式开始,检查 Lib/test/test_importlib/test_abc.py 第 870-871 行中受影响的断言。跟踪 marshal.dumps 如何决定 FLAG_REF,然后编写测试以证明输出与调用方的引用计数无关,并确认 importlib 测试通过。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- c, python
- 领域
- backend
- Issue 类型
- 缺陷
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100