Do Not Try To Install Benchmark Dependencies If Already Done
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 203
- Avg merge
- 1h 20m
- Merged PRs (30d)
- 2
Description
Currently we install each benchmark's requirements every time we run the benchmarks, which adds to the time it takes. If we know they are already installed then we can skip that.
Possible solutions:
- only install requirements when a venv is created
- for the common venv, we'd need to track (in a file) which benchmarks have been done already
- only install requirements if not done already for the venv
- track with a file (e.g.
<venvroot>/.installed) - use a file to track what has already been installed in the venv
- use
pip freeze(or similar) to identify what has been installed already (requires the ability to verify requirements)
- track with a file (e.g.
All that said, I'm not sure this alone is worth the effort. It might be worth it if combined with #157.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing where benchmark requirements are installed and how the common and per-benchmark virtual environments are created. Compare tracking installed requirements with checking the environment, including the interaction with issue #157. Done means repeated benchmark runs skip work when dependencies are already available without compromising dependency correctness.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- performance, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100