python / python/cpython

`METH_METHOD` calling convention is now not so efficient

未關閉
#123,500 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

performance topic-C-API type-bug
主要語言
Python
星號
77.2k
分支
36k
PR 合併指標
PR 指標待擷取

描述

Bug report

Bug description:

There are callables implemented with the METH_METHOD|METH_FASTCALL signature in C. They can be 5%-15% less efficient than using only METH_FASTCALL (or METH_O) with a PyType_GetModuleByDef function call.

For example, I measured the difference on Windows PGO builds by duplicating functions:

  • CDataType_from_buffer_copy() in _ctypes.c, which is not called when profiling:

    from timeit import timeit
    setup = """if 1:
        import ctypes
        buf = bytearray(16)
        cls = ctypes.c_char * len(buf)
    """
    # with a warmup
    for _ in range(2):
        # METH_METHOD|METH_FASTCALL (as-is)
        r0 = timeit(s0 := f'cls.from_buffer_copy (buf)', setup)
    
        # METH_FASTCALL (no `defining_class`) + PyType_GetModuleByDef
        r1 = timeit(s1 := f'cls.from_buffer_copy1(buf)', setup)
    
    print(s0, r0, 1 + (1 - r0 / r0))
    print(s1, r1, 1 + (1 - r1 / r0))
    
    cls.from_buffer_copy (buf) 0.15552800190635024 1.0
    cls.from_buffer_copy1(buf) 0.13187471489945893 1.1520837837364741
    
  • dec_mpd_qquantize() in _decimal.c profiled with 6800 calls (unfair?):

    # legacy (as-is)
    d1.quantize (d2) 0.1694609627971658 1.0
    
    # METH_METHOD|METH_FASTCALL (`defining_class`) + _PyType_GetModuleState
    d1.quantize1(d2) 0.1408861404022900 1.168621857938327
    
    # METH_FASTCALL (no `defining_class`) + PyType_GetModuleByDef
    d1.quantize2(d2) 0.1258157708973158 1.257553074049807
    
    Script (expand)
    from timeit import timeit
    setup = """if 1:
        from _decimal import Decimal
        d1,d2 = Decimal(1.414), Decimal('0.01')
    """
    for _ in range(2):
        r0 = timeit(s0 := f'd1.quantize (d2)', setup)
        r1 = timeit(s1 := f'd1.quantize1(d2)', setup)
        r2 = timeit(s2 := f'd1.quantize2(d2)', setup)
    
    print(s0, r0, 1 + (1 - r0 / r0))
    print(s1, r1, 1 + (1 - r1 / r0))
    print(s2, r2, 1 + (1 - r2 / r0))
    

Observations:

  • The number of arguments had little to do with this.
  • The gaps seem to be consistent as long as they are equally (un)exercised.
  • The same goes for non-PGO builds and builtin modules (e.g. _sre), where the impacts may be less significant.
CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

研究方向

先重現 _ctypes.c 中 CDataType_from_buffer_copy() 和 _decimal.c 中 dec_mpd_qquantize() 的報告基準,將 METH_METHOD|METH_FASTCALL 與 METH_FASTCALL 以及所示的模組查找呼叫進行比較。閱讀 METH_METHOD 和 PyType_GetModuleByDef 進入點,然後定義並驗證一種可接受的方法,確保其效能在所引用的各個案例中不差於替代方案。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
c, python
領域
backend, performance
Issue 類型
缺陷
難度
5/5
預估耗時
一週以上
活躍度
停滯
描述清晰度
需要釐清
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。