python / python/pyperformance

Do Not Try To Install Benchmark Dependencies If Already Done

Open
#158 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1k
Forks
203
Avg merge
1h 20m
Merged PRs (30d)
2

Description

Currently we install each benchmark's requirements every time we run the benchmarks, which adds to the time it takes. If we know they are already installed then we can skip that.

Possible solutions:

  • only install requirements when a venv is created
    • for the common venv, we'd need to track (in a file) which benchmarks have been done already
  • only install requirements if not done already for the venv
    • track with a file (e.g. <venvroot>/.installed)
    • use a file to track what has already been installed in the venv
    • use pip freeze (or similar) to identify what has been installed already (requires the ability to verify requirements)

All that said, I'm not sure this alone is worth the effort. It might be worth it if combined with #157.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing where benchmark requirements are installed and how the common and per-benchmark virtual environments are created. Compare tracking installed requirements with checking the environment, including the interaction with issue #157. Done means repeated benchmark runs skip work when dependencies are already available without compromising dependency correctness.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.