Comfy-Org / Comfy-Org/ComfyUI_frontend
[test-recorder] Secret scrubbing misses multiline/unlabeled input; raw autosave written to disk unscrubbed
- Dominant language
- TypeScript
- Stars
- 2k
- Forks
- 699
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 490
Description
### Problem
The secret scrubber in `tools/test-recorder` has gaps, and raw (unscrubbed) recorded code is written to disk during recording:
1. `src/transform/scrub.ts:6-10,17-20,45-48` — scrubbing is line-based: each line must independently contain both a sensitive label and a typing call. A password recorded as a multiline locator/action chain, or typed into a field identified only by test ID, survives scrubbing. Short values also evade the long-token heuristic.
2. `src/recorder/template.ts:57-85,145-154` + `src/recorder/runner.ts:172-180` — during recording, Playwright continuously autosaves raw generated code (including any typed credentials) to `browser_tests/tests/_recorded-code.txt` in plaintext. Sanitization only happens after recording completes (`src/transform/engine.ts:24-25`).
3. Detection doesn't cover common API-key formats (e.g. `sk-`, `ghp_`, `AKIA` prefixes).
This already happened in live user testing: a typed cloud password landed in generated output.
### Suggested fix
- Make scrubbing structure-aware (associate locator chains with their fill/type actions across lines; conservatively redact typing into password-type inputs and credential-labelled locators).
- Scrub the autosave stream continuously, or write it to a private temp location with restrictive permissions.
- Add API-key-format patterns and an always-visible pre-record reminder that credentials should only be entered before pressing Record.
_Line references are against branch `cb/comfy-test-usability` (PR #15837); adjust after merge._
Contributor guide
Assessment
This issue has not been assessed yet.