Require the server case to validate its own generated process
- Dominant language
- Rust
- Stars
- 53
- Forks
- 16
- Avg merge
- 4h 22m
- Merged PRs (30d)
- 46
Description
## Reproduced false pass
[run_server_test](https://github.com/wavefnd/Wave/blob/ed149e7553c06efc30b5167920cc301563d6afb3/tools/run_tests.py#L291) launches the compiler, waits one second, connects to fixed port 8080, and accepts a fixed response substring. It never checks the launched compiler/process status or proves the response belongs to that test. This path is used by `tests/cases/linux/amd64/test5.wave`.
I called the actual helper with `[python, "-c", "import sys; sys.exit(7)"]` and an injected socket returning the expected welcome text. The helper returns `(1, None)` and prints `PASS (server responded)` even though the launched command fails. Socket injection makes this reproduction deterministic without taking over a local port.
A real pre-existing listener on 8080 can provide the same false success. Conversely, the fixed one-second wait includes compilation and can report connection refusal before a correct generated server is ready.
## Acceptance
- [ ] A compiler or launched-server failure cannot be accepted because another listener responds.
- [ ] Use bounded readiness tied to this test's process and endpoint, keeping compilation and startup failure evidence.
- [ ] Validate enough of the response/protocol to identify the intended test instance.
- [ ] Retain process-tree cleanup and legitimate environment-based socket skips.
- [ ] Test failed compiler plus unrelated response, delayed startup, and a healthy server.
This is a server-test validity/readiness defect found while auditing the CI harness. #507 concerns descendant cleanup, which does not establish response ownership; #455 concerns unavailable local sockets.
Contributor guide
Research direction
Start in tools/run_tests.py at run_server_test around line 291, then inspect the test path using tests/cases/linux/amd64/test5.wave. Reproduce the failed compiler, unrelated response, delayed startup, and healthy-server cases described in the issue. Done means the helper ties readiness and response validation to its launched process while preserving cleanup and legitimate socket skips.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- testing-qa, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 56/100