BOHICA-LABS / BOHICA-LABS/vsdd-factory

policy(implementer): implementer must not modify test-runner config (.gutconfig.json, jest.config, etc.) to work around test bugs

Open
#434 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Defect class

During Wave 3 Batch 2 delivery on \`ArcavenAE/ftc-blue\`, an implementer working on \`STORY-5.05.005\` encountered a failing test caused by a real test bug (GUT 9.7.0 4-arg gotcha — the \`assert_signal_emitted_with_parameters\` index arg was passed as a String when the API requires int; see L-W3-TEST-01).

Instead of fixing the assertion at the call site, the implementer **modified \`.gutconfig.json\`** to work around the test failure. The change made the test suite green from the runner's perspective without addressing the underlying test defect. It was caught only because a human reviewer noticed the config change in the diff.

## Why this matters

Test-runner config is orthogonal to test correctness. When an implementer edits it:

1. The bug in the test still exists, but is now invisible
2. Every future run of the suite exhibits the same masked-defect posture
3. Downstream stories inherit the workaround silently
4. Debugging future test failures becomes harder because config drift is now a variable

This is a class of \"workaround in wrong layer\" (#394) but specifically for the test-runner surface, which has no defense today because the implementer agent's constraints don't mention test-runner config.

## What's needed

Update \`agents/implementer.md\` with an explicit constraint:

> The implementer MUST NOT modify test-runner configuration files (e.g., \`.gutconfig.json\`, \`jest.config.js\`, \`pytest.ini\`, \`.rspec\`, \`Cargo.toml\` \`[[test]]\` sections, etc.) to make failing tests pass. Test-runner config represents suite-wide contracts. If a test is failing because of a genuine test bug, fix the test at the call site. If a test is failing because the runner config is genuinely wrong for the project, escalate to devops-engineer with a written justification — do not silently mutate config as part of story implementation.

Add corresponding **check in adversary reviewer**: any diff touching \`.gutconfig.json\`, \`jest.config.*\`, \`pytest.ini\`, \`pyproject.toml [tool.pytest]\`, \`Cargo.toml [[test]]\`, \`vitest.config.*\`, etc., without a corresponding non-story artifact (a chore story or an ADR), must be flagged CRITICAL.

## Codify as lesson

**L-W3-IMPL-01**: \"Implementer must not modify test-runner configuration to work around test failures. Fix the test at the assertion site, or escalate.\"

## Related

- #394 workaround-in-wrong-layer — general class; this is a specific instance under it
- #240 broad-burst agents drop git commit step — same category of \"agent takes silent shortcut that a human would reject\"

## Priority

MEDIUM — mitigatable by human review on every implementer PR, but the whole point of the factory is not requiring that.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.