Comfy-Org / Comfy-Org/ComfyUI_frontend

[test-recorder] Decide scope of the optional agent cleanup step: shrink surface vs rely on it as safety net

Open
#15,849 0 comments 0 reactions 1 assignee Claimed by @christian-byrne View on GitHub
area:testing enhancement question
Dominant language
TypeScript
Stars
2k
Forks
704
Avg merge
1d 8h
Merged PRs (30d)
512

Description

### Decision needed
The optional post-recording cleanup step (`src/agent/refactor.ts:1-158`, `src/checks/agentCli.ts:3-70`, `src/commands/record.ts:75-132,772-774`) got opposite reviewer verdicts:

- **Simplify/remove:** ~230 lines that detect five evolving third-party CLIs, grant workspace-write access, manage a 10-minute subprocess, and parse custom summary markers. The deterministic transform already produces a valid spec; maintainers can refine during review.
- **Keep/rename:** in live user testing this step demonstrably saved the session — it stripped a recorded password and cleaned an accidental double-recording into a shippable test. Deleting it removes the safety net that made the first non-dev contribution succeed. Suggested instead: rename user-facing copy from "Refactor" to "Optional AI cleanup" (`src/commands/record.ts:101-115`).

### Suggested resolution to evaluate
Keep the step but shrink its surface: support fewer CLI adapters, cap permissions, and move the secret-stripping duty into the deterministic scrubber (tracked separately) so the cleanup step is an enhancement rather than a safety requirement.

_Line references are against branch `cb/comfy-test-usability` (PR #15837); adjust after merge._

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.