[C-API Docs] Clarify Constraints on the Arena Allocator
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 77.2k
- 派生
- 36k
- PR 合并指标
- PR 指标待抓取
描述
The "arena" allocator is used by obmalloc, as well as the code for allocating frames. It is exposed by the following public C-API:
PyObject_GetArenaAllocator()PyObject_SetArenaAllocator()
(It is also exposed/used internally as _PyObject_VirtualAlloc() and _PyObject_VirtualFree(). The implementation is exposed internally as _PyMem_ArenaAlloc() and _PyMem_ArenaFree().)
First of all, the docs for PyObject_GetArenaAllocator() are not clear at all about what the function should be used for. I expect it is only meant for use in combination with PyObject_SetArenaAllocator() (e.g. to wrap the old one; to check what's set). That should be clarified.
Second, the C-API docs make it clear that the two public functions may be called before the runtime is initialized. However, the following should be clarified, as there are no documented restrictions AFAICS:
- may
PyObject_SetArenaAllocator()be called after runtime init?- status quo: presumably not, since obmalloc would break (similar to the restrictions on
PyMem_SetAllocator()) - internally: we do not ever change the arena allocator
- recommendation: explicitly document that it must not be called after runtime init (a la
PyMem_SetAllocator())
- status quo: presumably not, since obmalloc would break (similar to the restrictions on
- may the allocator returned by
PyObject_GetArenaAllocator()be used directly?- status quo: possibly yes, since people may be doing so at this point (unlikely though)
- internally: we use it directly (but that's kind of the point)
- recommendation: explicitly state that it should not be used directly (point to object allocator API instead)
- ...before runtime init?
- (relevant only if direct use is okay)
- status quo: possibly yes, since people may be doing so at this point (unlikely though)
- internally: we do not use it before runtime init
- recommendation: at the least document that it shouldn't be used after runtime init
- better: state that it's only allowed when
PyObject_SetArenaAllocator()is allowed (i.e. only before init)
- must
PyObject_GetArenaAllocator()be called with the GIL held after runtime init?- (relevant only if post-init
PyObject_SetArenaAllocator()is okay) - status quo: possibly not, since people haven't needed to before (unlikely though)
- internally: we always hold the GIL
- recommendation: recommend (or require) that the GIL be held
- (relevant only if post-init
- must the allocator passed to
PyObject_SetArenaAllocator()be thread-safe (and reentrant)?- status quo: if the allocator may be used directly then possibly yes (since it seems it can be used before runtime init, as well as later without the GIL held); otherwise no (since the PyMem "object" allocator is documented as protected by the GIL)
- internally: the default arena allocator is thread-safe (and reentrant)
- recommendation: recommend that the arena allocator be thread-safe (just like the PyMem "raw" allocator)
- better: explicitly require it (so we can later drop the restriction on the PyMem "mem" and "object" allocators)
If we do specify tighter restrictions than were already documented then we'll need to consider who might be using the public functions in unexpected ways. However, I'd be surprised if there were more than a few (if any) using them at all.
Regardless, ISTM it would be better to deprecate the two public functions and replace them with a new PyConfig field. That would align a lot better with the intention. If there is a need to see if the currently set allocator matches some expected one, then it would be better to have an explicit function for that (e.g. PyObject_IsArenaAllocator()) rather than exposing PyObject_GetArenaAllocator() for that purpose.
The same goes for PyMem_SetAllocator(), where we'd have an additional PyMem_SetAllocatorWrapper() for the after-init case.
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
先从链接的 C-API 文档开始,查看 PyObject_GetArenaAllocator()、PyObject_SetArenaAllocator() 以及 Python 初始化之前的指导,然后对比 PyMem_SetAllocator 的限制。确定并记录初始化、GIL、直接使用和线程安全方面允许的约束;分别评估是否应指定弃用以及使用 PyConfig 的替代方案。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- c, python
- 领域
- api, documentation
- Issue 类型
- 文档
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 25/100