Is beartype cost acceptable?

从 zimscraperlib/init.py 和 beartype_this_package() 调用开始，然后检查 zimscraperlib.rewriting 热路径以及文档中记录的 warc2zim 基准测试命令。如有需要，重新执行成对运行，并比较实际耗时、CPU 时间和峰值 RSS。完成的标准是就 coverage 是否应限定范围、使用成本更低的策略进行配置，或针对选定的运行禁用达成共识并完成记录。

Context

While investigating a memory leak (fixed in #324 — a closure in RxRewriter.rewrite() was being re-decorated by beartype's claw hook on every call, permanently pinning it via beartype's own is_object_blacklisted cache), the fix removed the leak but also raised the question of how much runtime cost beartype_this_package() (used project-wide in zimscraperlib/__init__.py) actually adds. This issue is to share those numbers and discuss whether the tradeoff is the one we want.

Methodology

Ran warc2zim to completion, twice, on the same input, same machine, back to back:

warc2zim --publisher=openZIM --content-header-bytes-length=2048 \
  --zim-file=www.physicsclassroom.com_56e21a6e.zim --name=www.physicsclassroom.com_56e21a6e \
  --scraper-suffix "zimit 3.1.2" --output output \
  --url https://www.physicsclassroom.com/ \
  <local WARC, 731MB compressed, ~14.5K ZIM entries> --overwrite

wrapped in /usr/bin/time -v for wall-clock, CPU, and peak RSS.

• Run A: zimscraperlib as-is (beartype_this_package() active).
• Run B: same install, with the single line beartype_this_package() in zimscraperlib/__init__.py commented out (everything else identical), bytecode cache cleared between runs.

Both runs post-date the #324 leak fix, so memory is no longer confounded by that bug.

Results

Both runs completed successfully with an identical entry count. Memory is essentially the same between the two (the small gap is within run-to-run noise).

Takeaway

Runtime type-checking via beartype_this_package() costs roughly 15-18% additional wall-clock/CPU time on this workload, with no meaningful memory cost of its own now that #324 is fixed.

Discussion

Is this tradeoff (broad runtime type-safety across zimscraperlib, for ~15-18% slower conversions) the one we want, or should beartype's coverage be scoped down — e.g. excluding hot paths like zimscraperlib.rewriting from beartype_this_package(), or using a cheaper BeartypeStrategy for those modules — while keeping full checking elsewhere, or allowing to fully disable beartype "on-demand" (e.g. enable it only for unit and e2e tests of scraperlib and scrapers)?