Base manifest doesn't allow benchmark groups except "default" benchmark group
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 203
- Avg merge
- 1h 20m
- Merged PRs (30d)
- 2
Description
Where applicable, please fill out the details below to help us troubleshoot the
issue that you are facing. 🍰
How to reproduce the issue
Using script:
python3 -m venv testvenv
source testvenv/bin/activate
pip install pyperformance
testvenv/bin/pyperformance run -b math
What you expected to happen
The tool will run all the math benchmarks producing output like:
[1/3] float...
.....................
float: Mean +- std dev: 48.3 ms +- 1.6 ms
[2/3] nbody...
.....................
nbody: Mean +- std dev: 46.6 ms +- 1.7 ms
[3/3] pidigits...
What actually happens
Collecting pyperformance
Using cached pyperformance-1.11.0-py3-none-any.whl.metadata (2.0 kB)
Collecting pyperf (from pyperformance)
Using cached pyperf-2.7.0-py3-none-any.whl.metadata (6.0 kB)
Collecting packaging (from pyperformance)
Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting psutil>=5.9.0 (from pyperf->pyperformance)
Using cached psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)
Using cached pyperformance-1.11.0-py3-none-any.whl (4.2 MB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Using cached pyperf-2.7.0-py3-none-any.whl (139 kB)
Using cached psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (290 kB)
Installing collected packages: psutil, packaging, pyperf, pyperformance
Successfully installed packaging-24.1 psutil-6.0.0 pyperf-2.7.0 pyperformance-1.11.0
WARNING:root:no benchmark named 'math'
Python benchmark suite 1.11.0
ERROR: No benchmark was run
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the issue with python3 -m venv testvenv, installing pyperformance, and running testvenv/bin/pyperformance run -b math. Trace the pyperformance run entry point and base-manifest handling to find why only the default benchmark group is accepted; done means the math group runs its three benchmarks instead of reporting that no benchmark was run.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100