Comfy-Org / Comfy-Org/ComfyUI_frontend
Billing e2e cannot run on a signed-in local build by construction; desktop has zero coverage
- Dominant language
- TypeScript
- Stars
- 2k
- Forks
- 704
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 512
Description
Follow-up from the incident-95 local-credits thread. This is the structural reason the P0 shipped undetected despite heavy billing test investment.
## Problem / Goal
**Auth mocking is coupled to cloud project routing.** `browser_tests/fixtures/ComfyPage.ts:566-568` seeds auth only for `@cloud`-tagged tests:
```ts
if (testInfo.tags.includes('@cloud')) { await comfyPage.cloudAuth.mockAuth() }
```
But `@cloud` is a *project-routing* tag: `playwright.config.ts:37-40` has the default `chromium` project do `grepInvert: /@mobile|@perf|@audit|@cloud/`, and `:80-84` routes `@cloud` to the `cloud` project, which CI feeds the `frontend-dist-cloud` artifact. So obtaining a signed-in user required the one tag that guarantees `isCloud === true`. No billing flow was ever exercised as a signed-in user on a local build.
https://github.com/Comfy-Org/ComfyUI_frontend/pull/14587 added `browser_tests/fixtures/localAuthFixture.ts`, which builds `ComfyPage` directly and mocks auth unconditionally — its docstring states it exists *because* of this coupling. That is a side door for one spec, not a fix.
**Distribution coverage.** Distributions are `desktop | localhost | cloud` (`src/platform/distribution/types.ts:6`). Playwright has **no desktop project** and CI ships only `frontend-dist` and `frontend-dist-cloud`. Cloud is well covered; localhost has a small number of billing specs; **desktop has zero**.
**Entry points not covered anywhere:** the out-of-credits precondition dialog (`useAccountPreconditionDialog.ts:40`) and `BillingStatusBanner.vue`. Cloud-only: the `?topup=1` deep link (`useTopUpUrlLoader.ts:87`) and the workspace popover's Add credits. `topUpCreditsDialog.spec.ts` opens the dialog via `page.evaluate`, which proves the component but bypasses every real entry point.
**Also worth fixing while in here:** `topUpDeepLink.spec.ts:89,116` assert `toBeHidden()` through a translated-string locator. That passes on zero matches, so a copy rewording silently turns both assertions into no-ops.
## Proposed Solution
- Decouple auth seeding from project routing: introduce an auth tag (or promote `localAuthFixture` into the standard fixture) so any project can request a signed-in user without implying `isCloud`.
- Relax the blanket `@cloud` exclusion on the local project, since features are increasingly visible locally when subscribed.
- Add a desktop Playwright project, or state explicitly that desktop is covered by `@comfyorg/desktop-ui` and record where.
- Cover the two uncovered entry points and give the deep link a local-distribution case.
- Replace translated-string `toBeHidden()` locators with testid-based ones.
## Acceptance Criteria
- [ ] A spec can obtain a signed-in user in the `chromium` project without carrying `@cloud`, using the shared fixture rather than a bespoke one.
- [ ] Every credits/top-up entry point has at least one spec, and each spec's distribution is explicit.
- [ ] Desktop billing coverage either exists in CI or has a documented owner and location.
- [ ] No billing assertion depends on a translated string for a negative (hidden/absent) check.
Contributor guide
Assessment
This issue has not been assessed yet.