Improving representative benchmarks for typing ecosystem
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 1k
- 派生
- 203
- 平均合并
- 1 小时 20 分钟
- 30 天内合并 PR
- 2
描述
Due to a current lack of representative macrobenchmarks, it is very difficult to decide on whether complex accelerators for some parts of typing are worth implementing in the future. Hence, I'm trying to upstream some benchmarks into pyperformance.
IMO, there are three main areas:
- Performance of static type checkers implemented in Python (e.g. mypy). (Fixed by #102)
- Performance of programs using types at runtime (e.g. pydantic, attrs, etc.).
- Runtime overhead of typed code vs fully untyped code.
For case 2, I plan to use one of pydantic's benchmarks here https://github.com/samuelcolvin/pydantic/tree/master/benchmarks, installed without compiled binaries.
Case 3 is very tricky because there are so many ways to use typing. I don't know how often people use certain features, whether they type-hint inside tight loops, etc. So I'm struggling to find a good benchmark. An idea: grabbing one of the existing pyperformance benchmarks, fully type-hinting it, then comparing the performance delta may work.
CC @JelleZijlstra, I would greatly appreciate hearing your opinion on this (especially for case 3). Maybe I can post this on typing-sig too if I need more help.
Afterword:
All 3 cases benefit from general CPython optimizations. But usually only 3. benefits greatly from typing module-only optimizations (with 1. maybe not improving much if at all, depending on implementation).
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
首先查看 issue 中描述的三个基准测试领域,以及其中链接的现有 pydantic 基准测试。将这些选项与当前的 pyperformance 基准测试套件进行比较,然后明确应包含哪些案例和基准测试程序。达成具有代表性的基准测试范围共识,并将选定的基准测试 upstream 后,即视为完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- performance
- Issue 类型
- 功能
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 25/100