oxidecomputer / oxidecomputer/humility
humility test suite could be less needlessly cruel
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 584
- Forks
- 63
- Avg merge
- 3h
- Merged PRs (30d)
- 2
Description
Running the Humility test suite is one of the most unkind things I've done to my poor laptop: it takes about five minutes to run, and pins every CPU core at 100% for the entire duration:
It seems like this is because the complexity of the test suite is multiplicative: it's the number of cores times the number of commands we run. This means that adding a new command increases the total size of the test suite by len(cores), and adding a new core increases the total size of the test suite by len(commands).
There's probably a way to make this a bit less of a brutal dev machine torture test: in at least some cases, such as counters, a command is only "interesting" to run on newer cores that contain the structures the command interprets. Running humility counters on any core except for the one I added specifically to test the counters will just result in humility error: no counters found. So, we probably don't need to run the several different counters commands in the test suite against every core we currently test with. If we added a notion of "only run this command against these cores" to the test framework, we could make it a little less painful to run.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the Humility test suite and locating the test framework that forms the command-by-core matrix. Examine how commands such as humility counters behave on different cores. Done means commands can be limited to relevant cores while the existing coverage remains intact and the suite avoids redundant runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100