Make RustPython benchmarks readable

未關閉
#5,742 3 則留言 1 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

評估

難度
4/5
預估耗時
3-5 天
新手友好度
25/100
Issue 類型
功能
描述清晰度
需要釐清
活躍度
停滯
技術堆疊
rust

研究方向

從 issue 中連結的 benchmarks 頁面開始,特別查看 MICROBENCHMARKS、EXECUTION 和 PARSE_TO_AST 區段,並找出這些圖表在哪裡產生。比較目前的標籤、排序、色彩和可讀性,然後在變更更廣泛的中繼資料或互動性之前,定義一組聚焦的改進項目。完成的標準是 benchmark 圖表始終清晰易讀,並能區分 CPython 和 RustPython。

由索引模型根據 Issue 內容生成。

描述

TL;DR: benchmarks are poorly readable and could be greatly improved. This is key element in convincing people of the soundness of RustPython so it should probably not be neglected IMHO.


The violin plots available here are not easily readable and their Y-axes labels are hardly readable at all because they got left-cut at some point. This is especially troublesome for the MICROBENCHMARKS section, for which it is impossible to tell RustPython from CPython.

This issue could be alleviated by doing the following:

  • Use a specific color for CPython and another one for RustPython (and keep this color pair consistent across all plots).
  • Always have CPython data on top and RustPython data on bottom (this is not consistent: in the EXECUTION tab, CPython is on top and RustPython on bottom, while in tab PARSE_TO_AST it is the other way around).
  • Only keep the name of the benchmark in the Y-axis labels, i.e. replace execution/mandelbrot.py/cpython by either Mandelbrot (and use a legend to indicate which color is which interpreter), or make a plot title saying Mandelbrot and use the Y-axis labels to tell whether it is CPython or RustPython.

In addition to these visual issues, some other improvements could be implemented:

  • Make the plots user-friendly using some interactive backend such as plotly.
  • Put hyperlinks to the benchmark script location / source-code, so that users can check what the benchmarks are actually doing.
  • In the same line of thought, add a small descriptive text about what the benchmark does / why it is relevant (for instance "benchmark X is particularly I/O intensive" or whatnot).
  • On top of the page, give the hash of the commit / version (possibly with release date to know at a glance if they're outdated or not) of both CPython and RustPython binaries that were used, whether they were recompiled with -o3 locally, as well as the machine specs (this would allow for meaningful comparison and reproducibility).

I think that benchmarks one of the key element that might convince anyone to switch from one interpreter to another (apart from functionalities / low-level bindings). Hence they should not be neglected.

If someone could point me to where these plots are generated, I'd be happy to help typesetting them / add further info (although I might need some technical support about why benchmark X is especially relevant or not).

主要語言
Rust
星號
22.4k
分支
1.5k
平均合併
15 小時 18 分鐘
30 天內合併 PR
172

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue,再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫,在一個分支上完成修改。
  4. 送出 Pull Request,並在描述裡引用這個 Issue 編號。

RustPython/RustPython 的其他 Issue

查看 RustPython/RustPython 的全部 Issue

相似的 Issue

更多 Rust Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。