BOHICA-LABS / BOHICA-LABS/vsdd-factory
process-gap(test-writer+formal-verifier): config guards comparing parsed numbers must enumerate the config parser's full numeric-literal space
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Pattern
A guard comparing a number parsed from a YAML config file was validated against decimal integer inputs only. The guard used a leading-digit-run extraction strategy (read characters until non-digit). YAML 1.1 supports numeric forms that are not plain decimal integers:
| Form | Example | Parses as |
|------|---------|-----------|
| Underscored integer | `3_000` | 3000 |
| Hexadecimal | `0xFF` | 255 |
| Octal | `0o77` | 63 |
| Sexagesimal (base-60) | `1:0:0` | 3600 |
| Float | `3.14` | 3.14 |
| Scientific | `1.5e3` | 1500.0 |
| Tagged integer | `!!int 3000` | 3000 |
The leading-digit-run extraction reads `3_000` as `3` (stops at `_`). With a threshold of, say, `100`, this turns a config value of `3_000` (which should be 3000, clearly above the threshold) into an extracted value of `3` — below the threshold — silently tightening the guard's behavior rather than raising an error.
## Impact
The guard produced a false-tighten instead of a false-pass (which, depending on the guard's role, can mean a constraint that was intended to be relaxed is silently applied more strictly). The defect survived all prior adversarial passes because the test matrix used only plain decimal forms.
## Metrics
- Defect class: leading-digit-run extraction misreads underscore-separated integers
- Post-fix test matrix: 23 numeric forms enumerated (decimal variants, hex, octal, sexagesimal, floats, scientific, tagged), zero escapes
- Prior test matrix: decimal-only, 100% pass — defect invisible
## Framework ask
1. **Test-writer checklist item:** when writing tests for any guard or validator that parses a numeric field from a config format (YAML, TOML, JSON, env), include at minimum: plain integer, underscore-separated integer, hexadecimal, float, scientific notation, and tagged/typed variants as supported by that format's parser. This list should be derived from the format spec, not from the author's intuition.
2. **Formal-verifier / adversarial lens:** for numeric-comparison guards, one lens rotation should ask "what does this guard do with each numeric-literal form the config parser accepts?" and enumerate them. This is a bounded exhaustion (YAML 1.1 has ~8-10 forms), not an infinite search.
3. **Story-writer / BC guidance:** any BC mandating a numeric threshold check against a config file should cite the config format's full numeric-literal taxonomy as a testing constraint, not assume plain decimal.
## Cross-refs
- #480 (hand-seeded fixtures violating atomic-production invariants mask guard-placement bugs — same fixture-coverage-gap class, different axis)
- #477 (tautological zero-assertion tests — orthogonal; this is about test input coverage, not assertion quality)
- #478 (BC mandates API without verifying it exists in pinned engine version — spec-authoring checklist gap; analogous "know your runtime" principle)
*(Framework-pattern detail only. From a private project running the factory.)*
Contributor guide
Assessment
This issue has not been assessed yet.