lincc-frameworks / lincc-frameworks/python-project-template
Fail asv workflows when performance deteriorates
Nobody has claimed this yet.
- Dominant language
- Jinja
- Stars
- 85
- Forks
- 16
- Avg merge
- 6h 18m
- Merged PRs (30d)
- 1
Description
Fail the ASV workflows if at least one of benchmarks deteriorates above a certain threshold (e.g. 150% / 200%).
asv-main.yml:
- Execute
asv runwithout|| true, allowing the command to exit with a non-zero return code - Submit benchmarks and deploy dashboard on gh-pages, if the previous command succeeded or not
asv-pr.yml:
- Execute
asv continuouswithout|| true, allowing the command to exit with a non-zero return code - Output the performance comment on the PR if the comparison was successful or not
asv-nightly.yml
- Execute
asv continuouswithout|| true, allowing the command to exit with a non-zero return code - Write the commit hash to cache, if the previous if the comparison was successful or not
More information about "on failure" executions here.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with .github/workflows/asv-main.yml, asv-pr.yml, and asv-nightly.yml, and inspect how each currently handles asv run or asv continuous failures. Update the workflow failure conditions while preserving the requested submit, dashboard, PR comment, and cache steps, then verify the workflows still perform those steps when comparisons fail.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github-actions, python
- Domain
- ci-cd, devops, performance
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100