Upstream various benchmarking changes
Open
Nobody has claimed this yet.
benchmark
upstream
- Dominant language
- Python
- Stars
- 80
- Forks
- 8
- Avg merge
- 21h 44m
- Merged PRs (30d)
- 5
Description
Right now we have a heavily modified fork of zarrs_tools. A couple of things need to be upstreamed:
- Environment management.. There is no command for how to create an env, what packages are needed etc.
- System specifics API for swapping out commands for timing and cache busting (and anything else that might be system-specific)
- Small Datasets + CI The benchmarks probably should run on CI at least in a small case. I'm had a lot of trouble getting them to work on both Rocky Linux and my ubuntu. Rocky Linux did not like the
Popen(I had to pass inshell=True) and ubuntu was complaining about CLI downcasting - Parallelization Currently, the
zarrs-pythonis parallelized by default so benchmark is a bit wonky and not really "fair." See this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing zarrs_tools benchmarking setup and the four requested areas: environment management, system-specific commands, small datasets in CI, and parallelization. Check the reported Popen and CLI downcasting behavior on Rocky Linux and Ubuntu, then verify that benchmarks run consistently and fairly in the small CI case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, performance, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100