psf / psf/pyperf

Waiting for runner's bench_func and bench_command functions to complete instead of receiving outputs individually?

Open
#141 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
962
Forks
106
Avg merge
2d 18h
Merged PRs (30d)
4

Description

I'm trying to utilize pyperf to benchmark some functions and save their results to a CSV of my formatting.
I'm using this example code from the documentation:

#!/usr/bin/env python3
import pyperf
import time


def func():
    time.sleep(0.001)

runner = pyperf.Runner()
runner.bench_func('sleep', func)

I want to benchmark the same function with mulitprocessing.pool.Pool / mulitprocessing.pool.ThreadPool / concurrent.futures.ProcessPoolExecutor / concurrent.futures.ThreadPoolExecutor and varying values for things like the number of CPU cores and the chunksize for map functions.

The issue is that assigning a variable to store the output of runner.bench_func and printing out that variable leads to an output like this:

<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.<Benchmark 'python_startup' with 1 runs>
.
python_startup: Mean +- std dev: 24.5 ms +- 0.4 ms
<Benchmark 'python_startup' with 21 runs>

Whereas I want to suppress this output and wait for all runs to complete before moving forward with the program.

Is there some other way of storing the results of a benchmark that I can't seem to find in the documentation? Or should is there a way to force the runner to wait for a benchmark to complete?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the documented pyperf.Runner and bench_func example, then inspect how benchmark results and console output are exposed. Compare the requested multiprocessing and executor use cases with the current runner behavior. Done means the issue's desired result collection and completion behavior are clearly supported or documented, with the unwanted intermediate output addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.