Make live tests required PR checks
- Dominant language
- Bicep
- Stars
- 3.6k
- Forks
- 830
- Avg merge
- 1d 21m
- Merged PRs (30d)
- 79
Description
Currently the live tests are optional checks, and they are skipped for PRs sent from forks. However, failures in the live tests are usually ignored blindly as the test credentials expire quite often. We should consider making them required checks instead to force people to go through the error messages. This can help prevent regressions like https://github.com/Azure/bicep/issues/6823.
One thing we need to figure out is how to make the checks pass silently for PRs sent from forks, since required checks cannot be skipped (https://github.community/t/feature-request-conditional-required-checks/16761).
Contributor guide
Research direction
Start by tracing how the live tests are configured as optional PR checks and how checks behave for pull requests from forks. Read the failure context in issue 6823 and determine how required checks could remain successful for fork PRs. Done means the live tests are required for applicable PRs, fork behavior is defined, and regressions are no longer silently ignored.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github
- Domain
- ci-cd, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100