python / python/cpython

PyLong_AsInt32/64() are slow c.f. PyLong_AsLongAndOverflow()

Open
#141,546 3 comments 0 reactions 1 assignee View on GitHub

@skirpichev is already working on this.

Since Nov 16, 2025.

interpreter-core performance topic-C-API type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

Proposal:

Currently both PyLong_AsInt32() and PyLong_AsInt64() are using PyLong_AsNativeBytes(), but PyLong_AsLongAndOverflow()/PyLong_AsLongLongAndOverflow() should be in general - faster. This fact is already used in the PyLong_Export():
https://github.com/python/cpython/blob/4885ecfbda4cc792691e5d488ef6cb09727eb417/Objects/longobject.c#L6875-L6884

I suggest using same code e.g. for PyLong_AsInt64(). Indeed, consider following METH_O module function:

static PyObject *
bar(PyObject *Py_UNUSED(module), PyObject *arg)
{
    long val;

    if (!PyLong_AsInt64(arg, &val)) {
        return PyBool_FromLong(val != 0);
    }
    PyErr_Clear();
    Py_RETURN_FALSE;
}

We have timings:

111: Mean +- std dev: 166 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 243 ns +- 1 ns
(1<<62)+11: Mean +- std dev: 229 ns +- 7 ns
(1<<128)+2: Mean +- std dev: 693 ns +- 11 ns
(1<<256)+3: Mean +- std dev: 697 ns +- 13 ns

With the patched PyLong_AsInt64():

111: Mean +- std dev: 170 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 173 ns +- 1 ns
(1<<62)+11: Mean +- std dev: 176 ns +- 5 ns
(1<<128)+2: Mean +- std dev: 614 ns +- 7 ns
(1<<256)+3: Mean +- std dev: 614 ns +- 6 ns

Unfortunately, overflow exception adds some extra overhead we can't avoid. Things are better for following method:

static PyObject *
baz(PyObject *Py_UNUSED(module), PyObject *arg)
{
    int err = 1;                                                                                                                                         
    long val = PyLong_AsLongAndOverflow(arg, &err);                            
                                                                               
    if (!err) {                                                                
        return PyBool_FromLong(val != 0);                                      
    }                                                                          
    Py_RETURN_FALSE;                                                           
}                                                                              
111: Mean +- std dev: 165 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 169 ns +- 5 ns
(1<<62)+11: Mean +- std dev: 170 ns +- 1 ns
(1<<128)+2: Mean +- std dev: 156 ns +- 1 ns
(1<<256)+3: Mean +- std dev: 156 ns +- 1 ns
Benchmark ref patch baz
111 166 ns 170 ns: 1.02x slower 165 ns: 1.01x faster
(1<<32)+1 243 ns 173 ns: 1.40x faster 169 ns: 1.44x faster
(1<<62)+11 229 ns 176 ns: 1.30x faster 170 ns: 1.35x faster
(1<<128)+2 693 ns 614 ns: 1.13x faster 156 ns: 4.44x faster
(1<<256)+3 697 ns 614 ns: 1.13x faster 156 ns: 4.46x faster
Geometric mean (ref) 1.18x faster 2.08x faster

So, probably we want an internal helper function with PyLong_AsLongAndOverflow-like API. It can be used in the PyLong_Export() and few other places, which use PyLong_AsInt64() and clear exception.

diff --git a/Objects/longobject.c b/Objects/longobject.c
index 43c0db753a0..4baa7ba39a5 100644
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -6815,7 +6815,21 @@ int PyLong_AsInt32(PyObject *obj, int32_t *value)
 
 int PyLong_AsInt64(PyObject *obj, int64_t *value)
 {
-    LONG_TO_INT(obj, value, "C int64_t");
+    int overflow;
+#if SIZEOF_LONG == 8
+    long x = PyLong_AsLongAndOverflow(obj, &overflow);
+#else
+    // Windows has 32-bit long, so use 64-bit long long instead
+    long long x = PyLong_AsLongLongAndOverflow(obj, &overflow);
+#endif
+    Py_BUILD_ASSERT(sizeof(x) == sizeof(int64_t));
+    *value = (int64_t)x;
+    if (overflow) {
+        PyErr_SetString(PyExc_OverflowError,
+                        "Python int too large to convert to C int64_t");
+        return -1;
+    }
+    return PyErr_Occurred() ? -1 : 0;
 }
 
 #define LONG_TO_UINT(obj, value, type_name) \
# bench.py
import pyperf
from foo import asint64 as f

runner = pyperf.Runner()
for z in ['111', '(1<<32)+1', '(1<<62)+11', '(1<<128)+2', '(1<<256)+3']:
    h = f"{z}"
    z = eval(z)
    runner.bench_func(h, f, z)
/* foo.c */

#include <Python.h>

static PyObject *
bar(PyObject *Py_UNUSED(module), PyObject *arg)
{
    long val;

    if (!PyLong_AsInt64(arg, &val)) {
        return PyBool_FromLong(val != 0);
    }
    PyErr_Clear();
    Py_RETURN_FALSE;
}

static PyMethodDef methods[] = {
    {"asint64", (PyCFunction)bar, METH_O},
    {NULL}
};

static struct PyModuleDef def = {
    PyModuleDef_HEAD_INIT,
    "foo",  /* m_name */
    NULL,           /* m_doc */
    -1,             /* m_size */
    methods,        /* m_methods */
};

PyMODINIT_FUNC
PyInit_foo(void)
{
    return PyModule_Create(&def);
}
Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.