python / python/cpython

PyLong_AsInt32/64() are slow c.f. PyLong_AsLongAndOverflow()

未关闭
#141,546 3 条评论 0 个 reaction 已指派 1 人 在 GitHub 查看

@skirpichev 已经在做这个了。

开始于 2025年11月16日。

interpreter-core performance topic-C-API type-feature
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Feature or enhancement

Proposal:

Currently both PyLong_AsInt32() and PyLong_AsInt64() are using PyLong_AsNativeBytes(), but PyLong_AsLongAndOverflow()/PyLong_AsLongLongAndOverflow() should be in general - faster. This fact is already used in the PyLong_Export():
https://github.com/python/cpython/blob/4885ecfbda4cc792691e5d488ef6cb09727eb417/Objects/longobject.c#L6875-L6884

I suggest using same code e.g. for PyLong_AsInt64(). Indeed, consider following METH_O module function:

static PyObject *
bar(PyObject *Py_UNUSED(module), PyObject *arg)
{
    long val;

    if (!PyLong_AsInt64(arg, &val)) {
        return PyBool_FromLong(val != 0);
    }
    PyErr_Clear();
    Py_RETURN_FALSE;
}

We have timings:

111: Mean +- std dev: 166 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 243 ns +- 1 ns
(1<<62)+11: Mean +- std dev: 229 ns +- 7 ns
(1<<128)+2: Mean +- std dev: 693 ns +- 11 ns
(1<<256)+3: Mean +- std dev: 697 ns +- 13 ns

With the patched PyLong_AsInt64():

111: Mean +- std dev: 170 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 173 ns +- 1 ns
(1<<62)+11: Mean +- std dev: 176 ns +- 5 ns
(1<<128)+2: Mean +- std dev: 614 ns +- 7 ns
(1<<256)+3: Mean +- std dev: 614 ns +- 6 ns

Unfortunately, overflow exception adds some extra overhead we can't avoid. Things are better for following method:

static PyObject *
baz(PyObject *Py_UNUSED(module), PyObject *arg)
{
    int err = 1;                                                                                                                                         
    long val = PyLong_AsLongAndOverflow(arg, &err);                            
                                                                               
    if (!err) {                                                                
        return PyBool_FromLong(val != 0);                                      
    }                                                                          
    Py_RETURN_FALSE;                                                           
}                                                                              
111: Mean +- std dev: 165 ns +- 1 ns
(1<<32)+1: Mean +- std dev: 169 ns +- 5 ns
(1<<62)+11: Mean +- std dev: 170 ns +- 1 ns
(1<<128)+2: Mean +- std dev: 156 ns +- 1 ns
(1<<256)+3: Mean +- std dev: 156 ns +- 1 ns
Benchmark ref patch baz
111 166 ns 170 ns: 1.02x slower 165 ns: 1.01x faster
(1<<32)+1 243 ns 173 ns: 1.40x faster 169 ns: 1.44x faster
(1<<62)+11 229 ns 176 ns: 1.30x faster 170 ns: 1.35x faster
(1<<128)+2 693 ns 614 ns: 1.13x faster 156 ns: 4.44x faster
(1<<256)+3 697 ns 614 ns: 1.13x faster 156 ns: 4.46x faster
Geometric mean (ref) 1.18x faster 2.08x faster

So, probably we want an internal helper function with PyLong_AsLongAndOverflow-like API. It can be used in the PyLong_Export() and few other places, which use PyLong_AsInt64() and clear exception.

diff --git a/Objects/longobject.c b/Objects/longobject.c
index 43c0db753a0..4baa7ba39a5 100644
--- a/Objects/longobject.c
+++ b/Objects/longobject.c
@@ -6815,7 +6815,21 @@ int PyLong_AsInt32(PyObject *obj, int32_t *value)
 
 int PyLong_AsInt64(PyObject *obj, int64_t *value)
 {
-    LONG_TO_INT(obj, value, "C int64_t");
+    int overflow;
+#if SIZEOF_LONG == 8
+    long x = PyLong_AsLongAndOverflow(obj, &overflow);
+#else
+    // Windows has 32-bit long, so use 64-bit long long instead
+    long long x = PyLong_AsLongLongAndOverflow(obj, &overflow);
+#endif
+    Py_BUILD_ASSERT(sizeof(x) == sizeof(int64_t));
+    *value = (int64_t)x;
+    if (overflow) {
+        PyErr_SetString(PyExc_OverflowError,
+                        "Python int too large to convert to C int64_t");
+        return -1;
+    }
+    return PyErr_Occurred() ? -1 : 0;
 }
 
 #define LONG_TO_UINT(obj, value, type_name) \
# bench.py
import pyperf
from foo import asint64 as f

runner = pyperf.Runner()
for z in ['111', '(1<<32)+1', '(1<<62)+11', '(1<<128)+2', '(1<<256)+3']:
    h = f"{z}"
    z = eval(z)
    runner.bench_func(h, f, z)
/* foo.c */

#include <Python.h>

static PyObject *
bar(PyObject *Py_UNUSED(module), PyObject *arg)
{
    long val;

    if (!PyLong_AsInt64(arg, &val)) {
        return PyBool_FromLong(val != 0);
    }
    PyErr_Clear();
    Py_RETURN_FALSE;
}

static PyMethodDef methods[] = {
    {"asint64", (PyCFunction)bar, METH_O},
    {NULL}
};

static struct PyModuleDef def = {
    PyModuleDef_HEAD_INIT,
    "foo",  /* m_name */
    NULL,           /* m_doc */
    -1,             /* m_size */
    methods,        /* m_methods */
};

PyMODINIT_FUNC
PyInit_foo(void)
{
    return PyModule_Create(&def);
}
Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。