Allocated size miscalculated in ASDL sequence constructor

In the macro generating ASDL sequence constructor in pycore_asdl.h, the extra allocated size is miscalculated as sizeof(TYPE *) * (size - 1), while the correct one should be sizeof(TYPE) * (size - 1):

#define GENERATE_ASDL_SEQ_CONSTRUCTOR(NAME, TYPE) \
asdl_ ## NAME ## _seq *_Py_asdl_ ## NAME ## _seq_new(Py_ssize_t size, PyArena *arena) \
{ \
    asdl_ ## NAME ## _seq *seq = NULL; \
    size_t n; \
    /* check size is sane */ \
    if (size < 0 || \
        (size && (((size_t)size - 1) > (SIZE_MAX / sizeof(void *))))) { \
        PyErr_NoMemory(); \
        return NULL; \
    } \
    n = (size ? (sizeof(TYPE *) * (size - 1)) : 0); \                       // <----------------------------- this line
    /* check if size can be added safely */ \
    if (n > SIZE_MAX - sizeof(asdl_ ## NAME ## _seq)) { \
        PyErr_NoMemory(); \
        return NULL; \
    } \
    n += sizeof(asdl_ ## NAME ## _seq); \
    seq = (asdl_ ## NAME ## _seq *)_PyArena_Malloc(arena, n); \
    if (!seq) { \
        PyErr_NoMemory(); \
        return NULL; \
    } \
    memset(seq, 0, n); \
    seq->size = size; \
    seq->elements = (void**)seq->typed_elements; \
    return seq; \
}

A sample struct definition and usage taken from the source code:

// pycore_asdl.h
#define asdl_seq_GET(S, I) _Py_RVALUE((S)->typed_elements[(I)])

// pycore_ast.h
typedef struct {
    _ASDL_SEQ_HEAD
    expr_ty typed_elements[1];
} asdl_expr_seq;

// Python-ast.c
GENERATE_ASDL_SEQ_CONSTRUCTOR(expr, expr_ty)

// ast.c - validate_exprs
asdl_expr_seq *exprs
expr_ty expr = asdl_seq_GET(exprs, i);

This shows that the stored elements have type TYPE (i.e. expr_ty), not TYPE *. So the extra allocated memory should be sizeof(TYPE) * (size - 1) instead of sizeof(TYPE *) * (size - 1).

This hasn't cause problem yet because in every uses of GENERATE_ASDL_SEQ_CONSTRUCTOR, TYPE is either int or some pointer, so sizeof(TYPE*)'s are not smaller than sizeof(TYPE). But it can cause confusion and potential bug in the future.

Linked PRs

• gh-129934

Inizia in Include/internal/pycore_asdl.h da GENERATE_ASDL_SEQ_CONSTRUCTOR e esamina il lavoro collegato in PR gh-129934. Verifica che il calcolo dell'allocazione corrisponda alla memoria per gli elementi tipizzati, quindi esegui la suite di test CPython pertinente o i test ASDL/AST per confermare che il costruttore rimanga corretto.