BOHICA-LABS / BOHICA-LABS/vsdd-factory
process-gap(adversary+verification): encoded-artifact verification commands must be validated in decoded/executable form, not by grep against the source artifact
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- Avg merge
- 6h 43m
- Merged PRs (30d)
- 29
Description
## Encoded-artifact verification gap — verification commands embedded in JSON must be validated in decoded form
Observed on a private project running the factory. Framework-pattern detail only.
### Pattern
A verification command was embedded inside a JSON artifact (an attestation plan). The artifact contained a grep pattern that would be applied to a live log at attestation time. During remediation of an unrelated defect, one of the tokens the grep expected was corrected — but the correction was verified by grepping the **source JSON file**, not by decoding the JSON and running the extracted command.
Concretely: the tokens used a Unicode character represented in JSON as `—` (an em dash). Of six token locations in the file, five held the raw UTF-8 byte sequence and rendered/matched correctly. The sixth — the one embedded in the operative grep command inside a JSON string — held the JSON escape form `—`. When the string was parsed and passed to `grep`, the escape decoded to the six literal characters `—` (a backslash, then `u`, `2`, `0`, `1`, `4`), which matched nothing in the live log.
Because the fix was verified by `grep '' plan.json`, the source-side grep succeeded (the raw form was present five times in the file), and the fix was declared correct. The operative form — the one that actually runs — was never executed and never matched.
### How it was caught
The orchestrator, in a fresh-context verification pass, wrote a small script that:
1. Loaded the JSON with a real parser.
2. Extracted the verification-command field.
3. Executed the extracted pattern against a **synthetic log** constructed to represent the real emission that would occur during attestation.
The synthetic log contained the expected token in its raw form. The extracted pattern matched zero lines. The gap was identified immediately.
### Related earlier finding in the same story
A prior remediation pass on the same file had "fixed" the grep pattern to cite a `BC-`-style tag. Verification confirmed the pattern matched — against the fixed pattern grep-ing its own source. When executed against real emission, the tag the code emits is a different string entirely; the grep would have false-PASSed against a broken baseline. This is the same class of gap (verify against source, not against decoded execution) with a different failure mode.
### Framework asks
**Any embedded/encoded verification command must be validated by decode-then-execute.** When a verification step lives inside an encoded artifact (JSON string, YAML block-scalar, base64 blob, template with quoted metacharacters), the validation contract is:
1. Load the artifact with a real parser (not text-grep).
2. Extract the verification command as it will be executed at attestation time.
3. Run it against a **synthetic positive fixture** (asserts it matches the expected form).
4. Run it against a **synthetic negative fixture** (asserts it does *not* match a plausible near-miss form).
Text-grep against the source artifact is insufficient because JSON/YAML/template escapes decode away from the human-visible form.
**Adversary lens rotation item.** Add "decode-then-execute verification of any embedded command" as a rotation point in adversary lens rotation (#462). At least one convergence pass on any story that ships or modifies an encoded artifact containing an executable command must include this lens.
### Cross-refs
- #494 (attestation quality — fabricated evidence text and presence-vs-change verification gap; this is a third axis: encoded-form evidence gap)
- #462 (adversary lens rotation)
- #497 (post-convergence fresh-context review — how this instance was caught)
- #467 (grep-verify handler — related but distinct: that issue is about verifying grep-hit against spec canonical values; this is about verifying the grep command itself decodes correctly)
*(Framework-pattern detail only. From a private project running the factory.)*
Contributor guide
Assessment
This issue has not been assessed yet.