aws / aws/aws-durable-execution-conformance-tests
[Feature]: Enable the CI to detect missing suites / case
- Dominant language
- Python
- Stars
- 6
- Forks
- 3
- Avg merge
- 22h 2m
- Merged PRs (30d)
- 24
Description
### Problem / motivation
Currently CI are set to run the test cases with each suite. If new suite / test cases are added in the conformance test suite, the CI cannot detect / run them automatically. Provide a way in the test runner to provide coverage info of the test suites
### Proposed solution
Add a coverage option for the test runner, which only provides the yaml coverage without running the test cases. This can be used by the CI to detect coverage gaps
### Alternatives considered
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start by locating the Python test runner and the CI jobs that invoke each conformance suite. Trace how suite and case coverage is represented in YAML, then define completion as a runner option that reports YAML coverage without executing cases and lets CI identify missing suites or cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ci-cd, testing-qa
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100