wavefnd / wavefnd/Wave

Include multi-file cases and reject empty suites in the FreeBSD VM runner

Open
#506 0 comments 0 reactions 0 assignees View on GitHub
bug good first issue needs testing
Dominant language
Rust
Stars
53
Forks
16
Avg merge
4h 22m
Merged PRs (30d)
46

Description

## Problem and code evidence

`tools/check_freebsd_sys.py` added in #502 discovers only `test*.wave` directly in a FreeBSD architecture directory. The regular case runner also supports `testN/main.wave` directories. Moving an integration test into that supported multi-file layout therefore silently drops it from FreeBSD VM execution.

At `e204bbebbbede0086fbb670d43a65b9cf38f0231`, the VM runner also has no empty-selection check. With no matching standalone files, it generates a shell loop with no cases and no result checks. This is a missing guard, not a claim that an empty-suite VM run was executed during this audit.

Local discovery reproduction, no VM required:

```python
from pathlib import Path
from tempfile import TemporaryDirectory
with TemporaryDirectory() as tmp:
suite = Path(tmp)
(suite / "test1").mkdir()
(suite / "test1/main.wave").write_text("fun main() -> i32 { return 0; }\n")
print(list(suite.glob("test*.wave"))) # [] despite one supported multi-file case
```

Start with the discovery/build/result loops in `tools/check_freebsd_sys.py` and compare `iter_test_entries` in `tools/run_tests.py`.

## Acceptance

- [ ] Discover both standalone `testN.wave` and multi-file `testN/main.wave` cases with deterministic numeric ordering.
- [ ] Preserve the logical case name and build the correct `main.o` for a directory case.
- [ ] Reject empty or invalid/ambiguous selections before launching external tools or a VM.
- [ ] Add temporary-directory Python tests covering standalone, multi-file, mixed and empty suites; no LLVM, QEMU or downloads needed.
- [ ] Keep the existing O0/O2 execution and per-case result checks.

This is a small runner-discovery follow-up suitable for a first contribution. #460 covers orphan manifest directories and #461 covers the case generator; neither covers this VM runner.

Contributor guide

Open the contributing guide

Research direction

Start with the discovery, build, and result loops in tools/check_freebsd_sys.py, then compare iter_test_entries in tools/run_tests.py. Add temporary-directory Python tests for standalone, multi-file, mixed, empty, and invalid or ambiguous selections. Done means deterministic case discovery, correct directory main.o handling, early rejection before external tools or VM launch, and unchanged O0/O2 result checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
82/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.