wavefnd / wavefnd/Wave

Reject orphan and unconfigured directories in the case manifest

Open
#460 0 comments 0 reactions 0 assignees View on GitHub
build good first issue needs testing
Dominant language
Rust
Stars
53
Forks
16
Avg merge
4h 22m
Merged PRs (30d)
46

Description

## Problem

The manifest validator added by #457 verifies every configured suite, but an accidentally misspelled or abandoned directory below `tests/cases/` can remain outside the manifest and silently receive no CI coverage.

## Scope

- Walk suite directories below `tests/cases/` and compare them with the directories derived from `cases.toml`.
- Ignore documented non-suite files and nested multi-file case directories.
- Report every orphan path in one actionable validation error.
- Add temporary-directory unit tests for valid, orphaned, and misspelled layouts.

## Completion criteria

- `case_manifest.py validate` fails when a new suite directory is not represented by the matrix.
- Multi-file cases such as `shared/test88/` remain valid.
- The error names the exact paths and explains how to register or remove them.

Related: #457

Contributor guide

Open the contributing guide

Research direction

Start in case_manifest.py with the existing validate command, then trace how cases.toml becomes the configured suite directories under tests/cases/. Run the current validation and inspect the existing unit-test setup before adding temporary-directory coverage for valid, orphaned, misspelled, and nested multi-file layouts. Done means validate reports all orphan paths with registration or removal guidance while accepting shared/test88/.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, testing, tooling
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.