python / python/cpython

array.array.fromlist() exposes uninitialized memory when an element's __index__ resizes the array

Open
#152,166 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

array.array.fromlist() preallocates n slots with array_resize(self, old_size + n) and then fills them by calling the type's setitem at an index recomputed from the live Py_SIZE(self) on every iteration:

// Modules/arraymodule.c  array_array_fromlist_impl
old_size = Py_SIZE(self);
if (array_resize(self, old_size + n) == -1)
    return NULL;
for (i = 0; i < n; i++) {
    PyObject *v = PyList_GET_ITEM(list, i);
    if ((*self->ob_descr->setitem)(self,
                    Py_SIZE(self) - n + i, v) != 0) {   // <-- live Py_SIZE(self)
        ...
    }
    if (n != PyList_GET_SIZE(list)) {                   // guards the *list*, not self
        ...
    }
}

The loop guards against the source list changing size, but not against self being resized as a side effect of converting an element. Conversion goes through setitem_PyNumber_Index__index__, which can run arbitrary Python. If __index__ grows self, Py_SIZE(self) increases, the write index Py_SIZE(self) - n + i slides forward, and the slots reserved by the initial array_resize are never written. array_resize uses PyMem_RESIZE (realloc) with no zeroing, so those slots hold uninitialized heap memory — fully readable from Python after a successful (no-exception) return.

Reproduction:
import array
a = array.array('i')
class Evil:
    def __index__(self):
        a.extend([0]*5)   # grow self during the conversion callback
        return 111
a.fromlist([Evil(), 222, 333])
print(a.tolist())

On a debug build:

[111, -842150451, -842150451, 0, 0, 0, 222, 333]
#      ^^^^^^^^^^^  ^^^^^^^^^^  slots reserved by array_resize but never written

-842150451 is 0xCDCDCDCD, pymalloc's PYMEM_CLEANBYTE fill for allocated-but-unwritten memory, i.e. the slots are uninitialized. On a release build these slots contain arbitrary process heap bytes (information disclosure), and the real elements (222, 333) are misplaced into the wrong positions. A shrinking __index__ (e.g. del a[0]) likewise leaves an uninitialized slot exposed.

Relationship to gh-144128:

Same entry point as gh-144128 ("CPython UaF during index callbacks", fixed in gh-144138) but a distinct defect. gh-144138 fixed a use-after-free of the borrowed item reference by adding Py_INCREF/Py_DECREF around _PyNumber_Index in II/LL/QQ_setitem; it did not change array_array_fromlist_impl, so this index-recompute / uninitialized-slot issue is still present on main.

Suggested fix:

Fill the fixed slot old_size + i (rather than an offset recomputed from the live Py_SIZE) and add a guard that bails with RuntimeError("array changed size during iteration") if self is resized mid-iteration, mirroring the existing list-mutation guard. PR to follow.

CPython versions tested on:

main (3.16); the code is long-standing and earlier versions are affected too.

Linked PRs
  • gh-152167

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in Modules/arraymodule.c at array_array_fromlist_impl and reproduce the issue with the provided index callback that resizes the array. Check the existing mutation guard and the linked PR gh-152167. Done means resizing during conversion cannot expose unwritten slots or misplace elements, with regression coverage for growth and shrinkage.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
backend, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.