wavefnd / wavefnd/Wave

Reject missing object outputs in manifest compile-only case runs

Open
#514 0 comments 0 reactions 0 assignees View on GitHub
bug build good first issue needs testing
Dominant language
Rust
Stars
53
Forks
16
Avg merge
4h 22m
Merged PRs (30d)
46

Description

## Problem and reproduction

A manifest-selected compile-only lane can report PASS when the compiler exits zero without producing any object.

`tools/run_tests.py::command_for_test` requests `--emit=obj` for `manifest_compile_target()`. In `run_and_classify`, however, artifact validation runs only when `compile_target is None` (lines 405-412), so the cross-target branch checks the process exit code alone.

I executed the actual checked-in `run_and_classify` function in an isolated namespace with:

- a selected `aarch64-unknown-freebsd` compile target;
- ordinary successful metadata;
- `[sys.executable, "-c", "pass"]` as the fake compiler command.

The fake command creates no output. The function prints `PASS` and returns `(1, None)`.

## Scope and acceptance

- [ ] Require the expected object file to exist before a manifest compile-only case is counted as passed.
- [ ] Derive the output location from the same source/name/output-directory logic used to construct the command, including `testN/main.wave` cases.
- [ ] Add fake-compiler tests for exit-zero/missing-file, nonzero exit and a successful output.
- [ ] Use existing artifact checks where applicable and preserve host/target selection.
- [ ] Keep adding new object-format validators out of this small fix; #463 already tracks WebAssembly format validation.

No LLVM or native cross-platform machine is needed for the focused Python tests. This is a good first issue in test result classification, separate from #506's VM discovery and #462's report metadata.

Audited on canonical master `0c67f4cc0c3946cbf708c11ef79db927ec8f054e` (same source tree as #502 head). Executable reproductions used Fedora Linux amd64, Wave `0.2.1-pre-beta-dev`, LLVM 21.1.8.

Contributor guide

Open the contributing guide

Research direction

Start in tools/run_tests.py with command_for_test and run_and_classify, then trace manifest_compile_target() and the existing artifact checks. Add focused fake-compiler tests covering exit-zero with no object, nonzero exit, and successful output, including testN/main.wave paths. Done means missing expected objects cannot pass while host/target selection remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
88/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.