[C-API Docs] Clarify Constraints on the Arena Allocator
還沒有人認領這個 Issue。
- 主要語言
- Python
- 星號
- 77.2k
- 分支
- 36k
- PR 合併指標
- PR 指標待擷取
描述
The "arena" allocator is used by obmalloc, as well as the code for allocating frames. It is exposed by the following public C-API:
PyObject_GetArenaAllocator()PyObject_SetArenaAllocator()
(It is also exposed/used internally as _PyObject_VirtualAlloc() and _PyObject_VirtualFree(). The implementation is exposed internally as _PyMem_ArenaAlloc() and _PyMem_ArenaFree().)
First of all, the docs for PyObject_GetArenaAllocator() are not clear at all about what the function should be used for. I expect it is only meant for use in combination with PyObject_SetArenaAllocator() (e.g. to wrap the old one; to check what's set). That should be clarified.
Second, the C-API docs make it clear that the two public functions may be called before the runtime is initialized. However, the following should be clarified, as there are no documented restrictions AFAICS:
- may
PyObject_SetArenaAllocator()be called after runtime init?- status quo: presumably not, since obmalloc would break (similar to the restrictions on
PyMem_SetAllocator()) - internally: we do not ever change the arena allocator
- recommendation: explicitly document that it must not be called after runtime init (a la
PyMem_SetAllocator())
- status quo: presumably not, since obmalloc would break (similar to the restrictions on
- may the allocator returned by
PyObject_GetArenaAllocator()be used directly?- status quo: possibly yes, since people may be doing so at this point (unlikely though)
- internally: we use it directly (but that's kind of the point)
- recommendation: explicitly state that it should not be used directly (point to object allocator API instead)
- ...before runtime init?
- (relevant only if direct use is okay)
- status quo: possibly yes, since people may be doing so at this point (unlikely though)
- internally: we do not use it before runtime init
- recommendation: at the least document that it shouldn't be used after runtime init
- better: state that it's only allowed when
PyObject_SetArenaAllocator()is allowed (i.e. only before init)
- must
PyObject_GetArenaAllocator()be called with the GIL held after runtime init?- (relevant only if post-init
PyObject_SetArenaAllocator()is okay) - status quo: possibly not, since people haven't needed to before (unlikely though)
- internally: we always hold the GIL
- recommendation: recommend (or require) that the GIL be held
- (relevant only if post-init
- must the allocator passed to
PyObject_SetArenaAllocator()be thread-safe (and reentrant)?- status quo: if the allocator may be used directly then possibly yes (since it seems it can be used before runtime init, as well as later without the GIL held); otherwise no (since the PyMem "object" allocator is documented as protected by the GIL)
- internally: the default arena allocator is thread-safe (and reentrant)
- recommendation: recommend that the arena allocator be thread-safe (just like the PyMem "raw" allocator)
- better: explicitly require it (so we can later drop the restriction on the PyMem "mem" and "object" allocators)
If we do specify tighter restrictions than were already documented then we'll need to consider who might be using the public functions in unexpected ways. However, I'd be surprised if there were more than a few (if any) using them at all.
Regardless, ISTM it would be better to deprecate the two public functions and replace them with a new PyConfig field. That would align a lot better with the intention. If there is a need to see if the currently set allocator matches some expected one, then it would be better to have an explicit function for that (e.g. PyObject_IsArenaAllocator()) rather than exposing PyObject_GetArenaAllocator() for that purpose.
The same goes for PyMem_SetAllocator(), where we'd have an additional PyMem_SetAllocatorWrapper() for the after-init case.
貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
研究方向
先從連結的 C-API 文件開始,查看 PyObject_GetArenaAllocator()、PyObject_SetArenaAllocator() 以及 Python 初始化前的指引,接著比較 PyMem_SetAllocator 的限制。確定並記錄初始化、GIL、直接使用和執行緒安全方面允許的約束;另外評估是否應指定棄用以及使用 PyConfig 的替代方案。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- c, python
- 領域
- api, documentation
- Issue 類型
- 文件
- 難度
- 5/5
- 預估耗時
- 一週以上
- 活躍度
- 停滯
- 描述清晰度
- 需要釐清
- 新手友好度
- 25/100