make marshal output not dependent on reference count
還沒有人認領這個 Issue。
- 主要語言
- Python
- 星號
- 77.2k
- 分支
- 36k
- PR 合併指標
- PR 指標待擷取
描述
Bug report
Currently the marshal module will emit a previously-unseen object flagged as a potential reference from later objects, unless the object has a reference count of 1. See https://github.com/python/cpython/blob/b27b57c6e44a276c8a9843fd37d4cf65b2827d5c/Python/marshal.c#L305
This is an overly-conservative heuristic -- it's easy to construct cases where an object has a reference count >1 but is not actually referenced by any other object about to be marshaled, so FLAG_REF is set when it does not need to be.
This makes marshal output unstable depending on accidents of reference counting behavior in the code calling marshal.dumps.
I ran into this because the Cinder JIT is able to reduce unnecessary increfs, and that resulted in some importlib tests failing on comparison of marshal output at https://github.com/python/cpython/blob/b27b57c6e44a276c8a9843fd37d4cf65b2827d5c/Lib/test/test_importlib/test_abc.py#L870-L871 because under Cinder JIT the reference count of code_object in that method is 1.
This previously caused issues in distutils reproducibility, resulting in a partial fix that applies only to interned strings: https://github.com/python/cpython/pull/8226
It would be better if marshal would actually determine which objects have multiple parents in the DAG and deterministically use FLAG_REF or not based on that.
貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
研究方向
從 Python/marshal.c 中提到的參照計數啟發式開始,檢查 Lib/test/test_importlib/test_abc.py 第 870-871 行中受影響的判斷。追蹤 marshal.dumps 如何決定 FLAG_REF,接著建立測試以證明輸出與呼叫方的參照計數無關,並確認 importlib 測試通過。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- c, python
- 領域
- backend
- Issue 類型
- 缺陷
- 難度
- 5/5
- 預估耗時
- 一週以上
- 活躍度
- 停滯
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100