aws / aws/aws-durable-execution-conformance-tests

[Feature]: Enable the CI to detect missing suites / case

Open
#31 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
6
Forks
3
Avg merge
22h 2m
Merged PRs (30d)
24

Description

### Problem / motivation

Currently CI are set to run the test cases with each suite. If new suite / test cases are added in the conformance test suite, the CI cannot detect / run them automatically. Provide a way in the test runner to provide coverage info of the test suites

### Proposed solution

Add a coverage option for the test runner, which only provides the yaml coverage without running the test cases. This can be used by the CI to detect coverage gaps

### Alternatives considered

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by locating the Python test runner and the CI jobs that invoke each conformance suite. Trace how suite and case coverage is represented in YAML, then define completion as a runner option that reports YAML coverage without executing cases and lets CI identify missing suites or cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing-qa
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.