BOHICA-LABS / BOHICA-LABS/vsdd-factory

enhancement(test-writer): warn about GUT 9.x assert_signal_emitted_with_parameters 4th-arg signature trap

Open
#280 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Summary

The `test-writer` agent on Godot/GUT projects repeatedly generates calls of the form:

```gdscript
assert_signal_emitted_with_parameters(EventBus, "my_signal", [args], "human-readable message")
```

This pattern is intuitive (the 4th arg "looks like" a message string, mirroring assertion APIs in pytest/jest/etc.). But GUT 9.7's signature treats the 4th argument as an INTEGER emission index. Passing a String causes a runtime error inside `signal_watcher.gd`: `Invalid operands 'String' and 'int' in operator '=='`. The test is reported as FAILED even though the assertion would have passed semantically.

## Repro / observed pattern

1. Spawn test-writer for a story whose ACs involve EventBus signal emission.
2. Test-writer writes test functions using `assert_signal_emitted_with_parameters(EventBus, "signal_name", [expected_args], "message describing what's being verified")`.
3. Test fails at runtime with operator-mismatch error. Test-writer reports tests as "failing as expected for Red Gate" — the failures look semantic but are actually GUT-API-misuse.
4. Implementer (next dispatch) discovers the issue while making tests pass. Implementer fixes the test file by removing the 4th-arg message strings. This is technically scope-creep (implementer touching test files) but the API misuse is unambiguous.

This has happened on three separate stories I've observed. Each time the test-writer wrote the pattern, each time the implementer cleaned it up.

## Why this matters

- Tests that "fail for the wrong reason" mask real test-quality issues. A test that should fail because of a semantic assertion mismatch but ACTUALLY fails because of a GUT-API runtime error gives a false-positive Red Gate signal.
- Implementer scope-creep onto the test file is itself a separate concern (see related observation: implementer scope-creep on test files has no protocol).
- The pattern is the result of test-writers reasoning from generic test-framework experience, not from GUT's actual signature.

## Suggested mitigation

Pick one or more:

1. **test-writer agent prompt update:** add an explicit warning section for Godot/GUT projects: "`assert_signal_emitted_with_parameters(emitter, signal, params)` — DO NOT pass a 4th string argument; GUT treats the 4th arg as an emission index (int)."
2. **Project-level helper wrapper:** propose a default `gut_helpers.gd` template that wraps the GUT assertion in a more conventional `(emitter, signal, params, message)` signature for projects that prefer that ergonomics.
3. **Static check skill:** a `validate-test-conventions` skill that greps `.gd` test files for the misuse pattern and flags it before commit.

## Related

- #266 (per-story-delivery Rust-idiom hardcoding) — sibling issue about test-tooling assumptions
- #268 (stub-architect parser-not-just-lint) — sibling about test-tool gotchas

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.