BOHICA-LABS / BOHICA-LABS/vsdd-factory

process-gap(test-writer+formal-verifier): config guards comparing parsed numbers must enumerate the config parser's full numeric-literal space

Open
#498 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
2
Forks
1
Avg merge
6h 43m
Merged PRs (30d)
29

Description

## Pattern

A guard comparing a number parsed from a YAML config file was validated against decimal integer inputs only. The guard used a leading-digit-run extraction strategy (read characters until non-digit). YAML 1.1 supports numeric forms that are not plain decimal integers:

| Form | Example | Parses as |
|------|---------|-----------|
| Underscored integer | `3_000` | 3000 |
| Hexadecimal | `0xFF` | 255 |
| Octal | `0o77` | 63 |
| Sexagesimal (base-60) | `1:0:0` | 3600 |
| Float | `3.14` | 3.14 |
| Scientific | `1.5e3` | 1500.0 |
| Tagged integer | `!!int 3000` | 3000 |

The leading-digit-run extraction reads `3_000` as `3` (stops at `_`). With a threshold of, say, `100`, this turns a config value of `3_000` (which should be 3000, clearly above the threshold) into an extracted value of `3` — below the threshold — silently tightening the guard's behavior rather than raising an error.

## Impact

The guard produced a false-tighten instead of a false-pass (which, depending on the guard's role, can mean a constraint that was intended to be relaxed is silently applied more strictly). The defect survived all prior adversarial passes because the test matrix used only plain decimal forms.

## Metrics

- Defect class: leading-digit-run extraction misreads underscore-separated integers
- Post-fix test matrix: 23 numeric forms enumerated (decimal variants, hex, octal, sexagesimal, floats, scientific, tagged), zero escapes
- Prior test matrix: decimal-only, 100% pass — defect invisible

## Framework ask

1. **Test-writer checklist item:** when writing tests for any guard or validator that parses a numeric field from a config format (YAML, TOML, JSON, env), include at minimum: plain integer, underscore-separated integer, hexadecimal, float, scientific notation, and tagged/typed variants as supported by that format's parser. This list should be derived from the format spec, not from the author's intuition.

2. **Formal-verifier / adversarial lens:** for numeric-comparison guards, one lens rotation should ask "what does this guard do with each numeric-literal form the config parser accepts?" and enumerate them. This is a bounded exhaustion (YAML 1.1 has ~8-10 forms), not an infinite search.

3. **Story-writer / BC guidance:** any BC mandating a numeric threshold check against a config file should cite the config format's full numeric-literal taxonomy as a testing constraint, not assume plain decimal.

## Cross-refs

- #480 (hand-seeded fixtures violating atomic-production invariants mask guard-placement bugs — same fixture-coverage-gap class, different axis)
- #477 (tautological zero-assertion tests — orthogonal; this is about test input coverage, not assertion quality)
- #478 (BC mandates API without verifying it exists in pinned engine version — spec-authoring checklist gap; analogous "know your runtime" principle)

*(Framework-pattern detail only. From a private project running the factory.)*

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.