wavefnd / wavefnd/Wave

Validate QEMU smoke cases against their target suites

Open
#646 0 comments 0 reactions 0 assignees View on GitHub
bug good first issue needs testing
Dominant language
Rust
Stars
53
Forks
16
Avg merge
4h 22m
Merged PRs (30d)
46

Description

## Current evidence

`tools/case_manifest.py` derives each target's selected suites as `shared`, `shared/`, and `/`. `smoke_case`, however, is currently validated only as a safe path to an existing file under `tests/cases`.

A QEMU target can therefore name a smoke case from another operating system or architecture and still pass manifest validation. The mistake is only exposed later when CI executes the generated matrix.

## Scope

- Require each `smoke_case` to belong to one of the suites selected for that target.
- Keep the existing safe-path and file-existence checks.
- Preserve valid shared, architecture-shared, and target-specific smoke cases.
- Report the target id and invalid smoke-case path in the manifest error.
- Keep runtime portability selection tracked by #459 separate from this manifest-integrity check.

## Completion criteria

- Unit tests accept smoke cases from valid selected suites.
- Unit tests reject smoke cases from another OS or architecture even when the file exists.
- Existing valid case matrices are unchanged.

Contributor guide

Open the contributing guide

Research direction

In tools/case_manifest.py, trace how target suites are derived and how smoke_case paths are currently validated. Find and run the manifest unit tests first, then extend coverage so valid shared, architecture-shared, and target-specific cases remain accepted while mismatched OS or architecture cases are rejected with the target id and path in the error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
74/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.