wavefnd / wavefnd/Wave

Require the server case to validate its own generated process

Open
#580 0 comments 0 reactions 0 assignees View on GitHub
bug ci help wanted needs testing
Dominant language
Rust
Stars
53
Forks
16
Avg merge
4h 22m
Merged PRs (30d)
46

Description

## Reproduced false pass

[run_server_test](https://github.com/wavefnd/Wave/blob/ed149e7553c06efc30b5167920cc301563d6afb3/tools/run_tests.py#L291) launches the compiler, waits one second, connects to fixed port 8080, and accepts a fixed response substring. It never checks the launched compiler/process status or proves the response belongs to that test. This path is used by `tests/cases/linux/amd64/test5.wave`.

I called the actual helper with `[python, "-c", "import sys; sys.exit(7)"]` and an injected socket returning the expected welcome text. The helper returns `(1, None)` and prints `PASS (server responded)` even though the launched command fails. Socket injection makes this reproduction deterministic without taking over a local port.

A real pre-existing listener on 8080 can provide the same false success. Conversely, the fixed one-second wait includes compilation and can report connection refusal before a correct generated server is ready.

## Acceptance

- [ ] A compiler or launched-server failure cannot be accepted because another listener responds.
- [ ] Use bounded readiness tied to this test's process and endpoint, keeping compilation and startup failure evidence.
- [ ] Validate enough of the response/protocol to identify the intended test instance.
- [ ] Retain process-tree cleanup and legitimate environment-based socket skips.
- [ ] Test failed compiler plus unrelated response, delayed startup, and a healthy server.

This is a server-test validity/readiness defect found while auditing the CI harness. #507 concerns descendant cleanup, which does not establish response ownership; #455 concerns unavailable local sockets.

Contributor guide

Open the contributing guide

Research direction

Start in tools/run_tests.py at run_server_test around line 291, then inspect the test path using tests/cases/linux/amd64/test5.wave. Reproduce the failed compiler, unrelated response, delayed startup, and healthy-server cases described in the issue. Done means the helper ties readiness and response validation to its launched process while preserving cleanup and legitimate socket skips.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
testing-qa, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
56/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.