Include multi-file cases and reject empty suites in the FreeBSD VM runner
- Dominant language
- Rust
- Stars
- 53
- Forks
- 16
- Avg merge
- 4h 22m
- Merged PRs (30d)
- 46
Description
## Problem and code evidence
`tools/check_freebsd_sys.py` added in #502 discovers only `test*.wave` directly in a FreeBSD architecture directory. The regular case runner also supports `testN/main.wave` directories. Moving an integration test into that supported multi-file layout therefore silently drops it from FreeBSD VM execution.
At `e204bbebbbede0086fbb670d43a65b9cf38f0231`, the VM runner also has no empty-selection check. With no matching standalone files, it generates a shell loop with no cases and no result checks. This is a missing guard, not a claim that an empty-suite VM run was executed during this audit.
Local discovery reproduction, no VM required:
```python
from pathlib import Path
from tempfile import TemporaryDirectory
with TemporaryDirectory() as tmp:
suite = Path(tmp)
(suite / "test1").mkdir()
(suite / "test1/main.wave").write_text("fun main() -> i32 { return 0; }\n")
print(list(suite.glob("test*.wave"))) # [] despite one supported multi-file case
```
Start with the discovery/build/result loops in `tools/check_freebsd_sys.py` and compare `iter_test_entries` in `tools/run_tests.py`.
## Acceptance
- [ ] Discover both standalone `testN.wave` and multi-file `testN/main.wave` cases with deterministic numeric ordering.
- [ ] Preserve the logical case name and build the correct `main.o` for a directory case.
- [ ] Reject empty or invalid/ambiguous selections before launching external tools or a VM.
- [ ] Add temporary-directory Python tests covering standalone, multi-file, mixed and empty suites; no LLVM, QEMU or downloads needed.
- [ ] Keep the existing O0/O2 execution and per-case result checks.
This is a small runner-discovery follow-up suitable for a first contribution. #460 covers orphan manifest directories and #461 covers the case generator; neither covers this VM runner.
Contributor guide
Research direction
Start with the discovery, build, and result loops in tools/check_freebsd_sys.py, then compare iter_test_entries in tools/run_tests.py. Add temporary-directory Python tests for standalone, multi-file, mixed, empty, and invalid or ambiguous selections. Done means deterministic case discovery, correct directory main.o handling, early rejection before external tools or VM launch, and unchanged O0/O2 result checks.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 82/100