NatLabRockies / NatLabRockies/H2Integrate
Assess how to reduce testing computational costs
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 26
- Forks
- 44
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 16
Description
Assess how to reduce testing computational costs
Right now, CI testing takes >30 mins, maybe more for Windows machiens.
It is fantastic that we have such exhaustive tests, but there's a chance we have some redundant or unnecessary tests that lead to unnecessarily inflated CI runtimes.
We should:
- assess if the existing tests can be intelligently combined to reduce computational burden
- potentially look at breaking up unit, regression, and integration tests into different CI pipelines so they run in parallel
- reduce size and scope of examples we test
- reduce model fidelity where reasonable
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing CI configuration and measuring runtimes for the unit, regression, and integration tests, including Windows runs. Assess redundant tests, example sizes, and model fidelity, then compare whether separate parallel pipelines reduce total computational cost without losing meaningful coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, performance, testing
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100