python / python/pyperformance

Base manifest doesn't allow benchmark groups except "default" benchmark group

Open
#356 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1k
Forks
203
Avg merge
1h 20m
Merged PRs (30d)
2

Description

Where applicable, please fill out the details below to help us troubleshoot the
issue that you are facing. 🍰

How to reproduce the issue
Using script:

python3 -m venv testvenv
source testvenv/bin/activate
pip install pyperformance
testvenv/bin/pyperformance run -b math

What you expected to happen
The tool will run all the math benchmarks producing output like:

[1/3] float...
.....................
float: Mean +- std dev: 48.3 ms +- 1.6 ms
[2/3] nbody...
.....................
nbody: Mean +- std dev: 46.6 ms +- 1.7 ms
[3/3] pidigits...

What actually happens

Collecting pyperformance
  Using cached pyperformance-1.11.0-py3-none-any.whl.metadata (2.0 kB)
Collecting pyperf (from pyperformance)
  Using cached pyperf-2.7.0-py3-none-any.whl.metadata (6.0 kB)
Collecting packaging (from pyperformance)
  Using cached packaging-24.1-py3-none-any.whl.metadata (3.2 kB)
Collecting psutil>=5.9.0 (from pyperf->pyperformance)
  Using cached psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (21 kB)
Using cached pyperformance-1.11.0-py3-none-any.whl (4.2 MB)
Using cached packaging-24.1-py3-none-any.whl (53 kB)
Using cached pyperf-2.7.0-py3-none-any.whl (139 kB)
Using cached psutil-6.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (290 kB)
Installing collected packages: psutil, packaging, pyperf, pyperformance
Successfully installed packaging-24.1 psutil-6.0.0 pyperf-2.7.0 pyperformance-1.11.0
WARNING:root:no benchmark named 'math'
Python benchmark suite 1.11.0



ERROR: No benchmark was run

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the issue with python3 -m venv testvenv, installing pyperformance, and running testvenv/bin/pyperformance run -b math. Trace the pyperformance run entry point and base-manifest handling to find why only the default benchmark group is accepted; done means the math group runs its three benchmarks instead of reporting that no benchmark was run.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.