python / python/cpython

PyLong_AsInt32/64() are slow c.f. PyLong_AsLongAndOverflow()

Abierto
#141,546 3 comentarios 0 reacciones 1 asignado Ver en GitHub

@skirpichev ya está trabajando en esto.

Desde el 16/11/2025.

interpreter-core performance topic-C-API type-feature
Lenguaje dominante
Python
Estrellas
77.2k
Forks
35.9k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

Feature or enhancement

Proposal:

Currently both PyLong_AsInt32() and PyLong_AsInt64() are using PyLong_AsNativeBytes(), but PyLong_AsLongAndOverflow()/PyLong_AsLongLongAndOverflow() should be in general - faster. This fact is already used in the PyLong_Export():
https://github.com/python/cpython/blob/4885ecfbda4cc792691e5d488ef6cb09727eb417/Objects/longobject.c#L6875-L6884

I suggest using same code e.g. for PyLong_AsInt64(). Indeed, consider following METH_O module function:

static PyObject *
bar(PyObject *Py_UNUSED(module), PyObject *arg)
{
    long val;

    if (!PyLong_AsInt64(arg, &val)) {
        return PyBool_FromLong(val != 0);
    }
    PyErr_Clear();
    Py_RETURN_FALSE;
}

We have timings:

111: Mean +- std dev: 166 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 243 ns +- 1 ns
(1<<62)+11: Mean +- std dev: 229 ns +- 7 ns
(1<<128)+2: Mean +- std dev: 693 ns +- 11 ns
(1<<256)+3: Mean +- std dev: 697 ns +- 13 ns

With the patched PyLong_AsInt64():

111: Mean +- std dev: 170 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 173 ns +- 1 ns
(1<<62)+11: Mean +- std dev: 176 ns +- 5 ns
(1<<128)+2: Mean +- std dev: 614 ns +- 7 ns
(1<<256)+3: Mean +- std dev: 614 ns +- 6 ns

Unfortunately, overflow exception adds some extra overhead we can't avoid. Things are better for following method:

static PyObject *
baz(PyObject *Py_UNUSED(module), PyObject *arg)
{
    int err = 1;                                                                                                                                         
    long val = PyLong_AsLongAndOverflow(arg, &err);                            
                                                                               
    if (!err) {                                                                
        return PyBool_FromLong(val != 0);                                      
    }                                                                          
    Py_RETURN_FALSE;                                                           
}                                                                              
111: Mean +- std dev: 165 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 169 ns +- 5 ns
(1<<62)+11: Mean +- std dev: 170 ns +- 1 ns
(1<<128)+2: Mean +- std dev: 156 ns +- 1 ns
(1<<256)+3: Mean +- std dev: 156 ns +- 1 ns
Benchmark ref patch baz
111 166 ns 170 ns: 1.02x slower 165 ns: 1.01x faster
(1<<32)+1 243 ns 173 ns: 1.40x faster 169 ns: 1.44x faster
(1<<62)+11 229 ns 176 ns: 1.30x faster 170 ns: 1.35x faster
(1<<128)+2 693 ns 614 ns: 1.13x faster 156 ns: 4.44x faster
(1<<256)+3 697 ns 614 ns: 1.13x faster 156 ns: 4.46x faster
Geometric mean (ref) 1.18x faster 2.08x faster

So, probably we want an internal helper function with PyLong_AsLongAndOverflow-like API. It can be used in the PyLong_Export() and few other places, which use PyLong_AsInt64() and clear exception.

diff --git a/Objects/longobject.c b/Objects/longobject.c
index 43c0db753a0..4baa7ba39a5 100644
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -6815,7 +6815,21 @@ int PyLong_AsInt32(PyObject *obj, int32_t *value)
 
 int PyLong_AsInt64(PyObject *obj, int64_t *value)
 {
-    LONG_TO_INT(obj, value, "C int64_t");
+    int overflow;
+#if SIZEOF_LONG == 8
+    long x = PyLong_AsLongAndOverflow(obj, &overflow);
+#else
+    // Windows has 32-bit long, so use 64-bit long long instead
+    long long x = PyLong_AsLongLongAndOverflow(obj, &overflow);
+#endif
+    Py_BUILD_ASSERT(sizeof(x) == sizeof(int64_t));
+    *value = (int64_t)x;
+    if (overflow) {
+        PyErr_SetString(PyExc_OverflowError,
+                        "Python int too large to convert to C int64_t");
+        return -1;
+    }
+    return PyErr_Occurred() ? -1 : 0;
 }
 
 #define LONG_TO_UINT(obj, value, type_name) \
# bench.py
import pyperf
from foo import asint64 as f

runner = pyperf.Runner()
for z in ['111', '(1<<32)+1', '(1<<62)+11', '(1<<128)+2', '(1<<256)+3']:
    h = f"{z}"
    z = eval(z)
    runner.bench_func(h, f, z)
/* foo.c */

#include <Python.h>

static PyObject *
bar(PyObject *Py_UNUSED(module), PyObject *arg)
{
    long val;

    if (!PyLong_AsInt64(arg, &val)) {
        return PyBool_FromLong(val != 0);
    }
    PyErr_Clear();
    Py_RETURN_FALSE;
}

static PyMethodDef methods[] = {
    {"asint64", (PyCFunction)bar, METH_O},
    {NULL}
};

static struct PyModuleDef def = {
    PyModuleDef_HEAD_INIT,
    "foo",  /* m_name */
    NULL,           /* m_doc */
    -1,             /* m_size */
    methods,        /* m_methods */
};

PyMODINIT_FUNC
PyInit_foo(void)
{
    return PyModule_Create(&def);
}
Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.