array.array.fromlist() exposes uninitialized memory when an element's __index__ resizes the array
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
array.array.fromlist() preallocates n slots with array_resize(self, old_size + n) and then fills them by calling the type's setitem at an index recomputed from the live Py_SIZE(self) on every iteration:
// Modules/arraymodule.c array_array_fromlist_impl
old_size = Py_SIZE(self);
if (array_resize(self, old_size + n) == -1)
return NULL;
for (i = 0; i < n; i++) {
PyObject *v = PyList_GET_ITEM(list, i);
if ((*self->ob_descr->setitem)(self,
Py_SIZE(self) - n + i, v) != 0) { // <-- live Py_SIZE(self)
...
}
if (n != PyList_GET_SIZE(list)) { // guards the *list*, not self
...
}
}
The loop guards against the source list changing size, but not against self being resized as a side effect of converting an element. Conversion goes through setitem → _PyNumber_Index → __index__, which can run arbitrary Python. If __index__ grows self, Py_SIZE(self) increases, the write index Py_SIZE(self) - n + i slides forward, and the slots reserved by the initial array_resize are never written. array_resize uses PyMem_RESIZE (realloc) with no zeroing, so those slots hold uninitialized heap memory — fully readable from Python after a successful (no-exception) return.
Reproduction:
import array
a = array.array('i')
class Evil:
def __index__(self):
a.extend([0]*5) # grow self during the conversion callback
return 111
a.fromlist([Evil(), 222, 333])
print(a.tolist())
On a debug build:
[111, -842150451, -842150451, 0, 0, 0, 222, 333]
# ^^^^^^^^^^^ ^^^^^^^^^^ slots reserved by array_resize but never written
-842150451 is 0xCDCDCDCD, pymalloc's PYMEM_CLEANBYTE fill for allocated-but-unwritten memory, i.e. the slots are uninitialized. On a release build these slots contain arbitrary process heap bytes (information disclosure), and the real elements (222, 333) are misplaced into the wrong positions. A shrinking __index__ (e.g. del a[0]) likewise leaves an uninitialized slot exposed.
Relationship to gh-144128:
Same entry point as gh-144128 ("CPython UaF during index callbacks", fixed in gh-144138) but a distinct defect. gh-144138 fixed a use-after-free of the borrowed item reference by adding Py_INCREF/Py_DECREF around _PyNumber_Index in II/LL/QQ_setitem; it did not change array_array_fromlist_impl, so this index-recompute / uninitialized-slot issue is still present on main.
Suggested fix:
Fill the fixed slot old_size + i (rather than an offset recomputed from the live Py_SIZE) and add a guard that bails with RuntimeError("array changed size during iteration") if self is resized mid-iteration, mirroring the existing list-mutation guard. PR to follow.
CPython versions tested on:
main (3.16); the code is long-standing and earlier versions are affected too.
Linked PRs
- gh-152167
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in Modules/arraymodule.c at array_array_fromlist_impl and reproduce the issue with the provided index callback that resizes the array. Check the existing mutation guard and the linked PR gh-152167. Done means resizing during conversion cannot expose unwritten slots or misplace elements, with regression coverage for growth and shrinkage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- backend, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100