Serenity: a new brand in a partially-migrated org must be born Semrush-backed

## Problem

An organization that migrates to Serenity brand by brand keeps creating **legacy (flat) brands** for as long as its migration is in flight. Every brand created during that window has to be migrated again by a later wave, and until then it produces LLMO-side prompts, schedules and DRS traffic that the wave then has to reconcile away.

Adobe Corp (`5d4e5082-b030-433d-9dbd-7007116f701f`) is in exactly that state today and is the **only** organization in prod that is: wave 1 released Adobe Helpx, Adobe Stock and Community Adobe on brand-scoped `feature_flags` rows, and the organization's own row is deliberately absent because 26 other brands would inherit it. A new brand created in that organization right now is a flat brand, and a create issued from a Serenity-mode client fails outright.

The desired behaviour: in an organization whose migration has started, a newly created brand is provisioned in Semrush and is live on the Serenity surfaces from the moment it exists — without taking the not-yet-migrated brands live with it.

## Verified current behaviour

Live prod `feature_flags` for Adobe Corp (2026-08-13): `LLMO/brandalf = true`, `LLMO/brandalf_migration = false`, and exactly three `LLMO/serenity` rows, each naming a brand (`019d8e58-6850-7ef0-98e7-abe3d0d83c2d` Adobe Helpx, `019d9041-166b-74b2-a829-2bfdcf7a4ef1` Adobe Stock, `a4a265fe-cb87-4e2d-8859-7d6f4f0aa365` Community Adobe), each written by the migration CLI and stamped `serenity_migration:`. No organization-scoped row. `organizations.semrush_workspace_id` is bound to the parent workspace `a827e263-fafb-49af-96f6-d3ef7fe2c33d`. The organization holds 16 active brands, of which exactly the 3 released ones carry a `semrush_sub_workspace_id`, plus 13 pending brands carrying none — so 13 active and 13 pending brands have no override.

Against `main` (`96fe4ddfd`, v1.741.1):

1. **Create mode asks the organization only.** `src/controllers/brands.js:1662` calls `isSerenityActiveForOrg`, which reads the organization's own row and ignores every brand override (`src/support/serenity/serenity-active.js:180`, `scopes?.orgRow?.flag_value === true`). For Adobe Corp that resolves `false`, so `isSemrushMode` is `false` and the create lands as a flat brand. A payload that actually asks for provisioning (`semrushMarket`, or `generatePrompts: true`) is rejected with `403 Serenity is not active for this organization` (`src/controllers/brands.js:1667`), which is what a Serenity-mode client sends.

2. **Forcing the mode alone would produce an invisible brand.** Every read and edit surface resolves per brand (`isSerenityActiveForBrand`, `src/support/serenity/serenity-active.js:209`) and a brand-new brand has no override row to resolve, so it falls back to the absent organization row and reads `false`: `src/controllers/elements.js:386` and `:435`, `src/controllers/serenity.js:415`, the brand-PATCH re-sync gate at `src/controllers/brands.js:1998`, and the `serenityActive` / `serenityActivatedAt` payload fields (`withSerenityState`, `src/support/brands-storage.js:210`) that project-elmo-ui renders through `isBrandSerenityActive`. The brand would hold a real sub-workspace and a real project while every surface served it as a classic brand. The deferred path is blocked the same way: `POST /serenity/activate` runs through the same per-brand gate in `authorize`, so a wizard draft saved as pending could never be activated.

3. **There is no write path for a brand-scoped flag row.** `upsertFeatureFlag` (`src/support/feature-flags-storage.js:179`) resolves the row to update with `.find(isOrgRow)` and hardcodes an insert payload of `{organization_id, product, flag_name, flag_value, updated_by}` — `brand_id` is never among the written columns. The only write endpoint, `PUT`/`DELETE /organizations/:organizationId/feature-flags/:product/:flagName` (`src/controllers/feature-flags.js:174`), has no brand path parameter and passes no brand id, and `FeatureFlagDto.toJSON` (`src/dto/feature-flag.js:21`) does not expose `brandId` at all. `listFeatureFlagsByOrgAndProduct` filters to organization rows (`src/support/feature-flags-storage.js:283`). The read half is fully built and wired — `readFeatureFlagScopes` partitions rows into `{orgRow, brandRows}` and `resolveFlagRowForBrand` prefers the brand's row — but with no writer in this service every row it sees carries `brand_id = null`, so it always resolves to the organization's row. The only writer of brand overrides today is the migration CLI (`mysticat-data-service` `scripts/serenity_migration/persist.py`).

## Why turning the organization row on is not the answer

The migration CLI's `--activate-only --org-id ` step writes the organization's own row and nothing else (`scripts/serenity_migration/__main__.py` `_activate_only` → `persist.py` `set_serenity_flag`, whose statements are all scoped `AND brand_id IS NULL`). There is no companion step that fences the unmigrated brands, so flipping that row publishes all 26 of them in the same instant — the precise outcome the absent row exists to prevent.

Even with a fence it would be the wrong instrument, because a fence is a snapshot and brand creation is not the only writer of `brands`:

- `src/controllers/llmo/llmo-onboarding.js:1413` creates a brand through `upsertBrand` with no Semrush provisioning at all. Any brand it writes after the flip inherits `serenity = true` with no sub-workspace, and lands broken: `404 Brand has no resolvable Semrush workspace` on the elements surfaces, and the classic prompt editors locked against it.
- The same holds for pending brands that a later flow promotes.
- The organization row also means "this customer is fully migrated" to the rest of the tooling — the planner's already-active branch, `customer_inventory`'s migration date — so flipping it early makes a partially migrated customer read as finished.

## Proposed change

Keep the organization row meaning "everything here is migrated", and give brand creation its own signal plus a stamp.

**1. Create-mode signal.** Decide Semrush-mode creates from the organization's migration state rather than its rollout row: the organization has a `semrush_workspace_id` **and** at least one brand-scoped `LLMO/serenity` row resolving `true`. Both halves are already read on this path — `readCachedFlagScopes` returns the organization's row and every brand override in the same cached entry, so the brand-row half costs no additional query — and the behaviour turns on for a customer exactly when its first wave lands. The alternative is an explicit organization flag (`serenity_new_brands` — snake_case is enforced by `feature_flags_flag_name_valid`), which is controllable but needs an ops step per customer and can be forgotten between waves.

**2. Stamp the brand on success.** When a create provisions a sub-workspace, write the brand's own `feature_flags` row `(organization_id, product='LLMO', flag_name='serenity', brand_id=) = true` in the same request, after the brand row exists (the composite FK `(organization_id, brand_id)` requires it). This makes the new brand live on every per-brand gate immediately, and leaves the organization row untouched, so no unmigrated brand inherits anything.

**3. Expose the signal for the wizard.** project-elmo-ui decides its create mode from `resolveSerenityMode` (`src/utils/devFlagOverrides.ts:153`), which reads the LD entitlement `FT_LLMO-5004`, the `serenity_ui` organization flag and the admin toggle — none of which are set for Adobe Corp. Setting `serenity_ui` is not an acceptable substitute: it is a superset gate that also locks the legacy prompt surfaces (LLMO-6694) for the 13 unmigrated brands. The organization payload should carry the new-brand signal so the wizard can select the Semrush create path per organization, independent of the UI-mode flag.

## What the change must also handle

Three consequences of the existing create path, none of which the stamp gets for free.

**The create response is built from a pre-write snapshot.** `readSerenityFlagScopes` is read at `src/controllers/brands.js:1825`, deliberately before `upsertBrand` so that a failure reading the rollout flag cannot report a brand that did get created as a 500. The 201 body is then `withSerenityState(created, serenityScopes)` (`:1874`), which resolves the new brand against that snapshot. A stamp written after `upsertBrand` is invisible to it, so the response would report `serenityActive: false` for a brand it just published. The stamped row has to be folded into the snapshot before it is used, rather than re-read.

**The flag cache holds the pre-stamp answer.** `serenity-active.js` caches `{orgRow, brandRows}` per organization for `BRAND_CACHE_TTL_MS`, so the container that served the create keeps resolving the new brand to the absent organization row until the entry expires — the brand 404s on the elements and `/serenity/*` surfaces during exactly the window in which the wizard redirects to it. The module exports only `clearSerenityFlagCache`, which is test-only and clears every organization; a targeted invalidation for the stamped organization is needed. Warm containers elsewhere still serve the stale answer until their own entry expires, which bounds the window rather than closing it.

**Stamp-failure compensation cannot restore a pre-create state exactly.** The composite FK forces the order brand row → stamp, so by the time a stamp can fail the brand row exists. The existing compensation in the create's `catch` (`src/controllers/brands.js:1890`) empties the just-provisioned sub-workspace and is gated on `provisionedWorkspaceWasCreated`, because an *adopted* workspace may be a same-named sibling brand's and is not ours to tear down. There is no matching brand-row compensation: `deleteBrand` (`src/support/brands-storage.js:1589`) is a soft delete that sets `status = 'deleted'`, so the row survives and keeps holding the `(organization_id, name)` unique slot.

## Work items

- [ ] `feature-flags-storage.js`: brand-scoped upsert alongside the organization-scoped one, keyed on the row id (the existing pattern), with the composite `(organization_id, brand_id)` FK respected. The wildcard `select('*')` projection that `isOrgRow` depends on must be preserved.
- [ ] `serenity-active.js`: the create-mode predicate, resolved from the cached scopes already read on this path, plus a targeted per-organization cache invalidation for use after a stamp.
- [ ] `brands.js` create: resolve the new create-mode signal in place of `isSerenityActiveForOrg`; on the Semrush path, stamp the brand's override after `upsertBrand`; fold the stamped row into the response snapshot; apply the agreed failure semantics.
- [ ] Pending (draft) creates: stamp at create time, so the later `POST /serenity/activate` passes `authorize` and can provision the stashed markets.
- [ ] Organization payload: expose the signal for project-elmo-ui. `OrganizationDto.toJSON` (`src/dto/organization.js:18`) is synchronous and is the serializer for `getAll` as well as the single-organization routes, so a derived asynchronous field needs the controller-side merge already used by `getSlackConfigByImsOrgID` (`src/controllers/organizations.js:209`), and the list route's per-organization fan-out cost has to be accounted for.
- [ ] Tests: unit coverage alongside `test/support/feature-flags-storage.test.js`, `test/support/serenity/serenity-active.test.js` and the `buildController` esmock blocks in `test/controllers/brands.test.js`; integration coverage with a brand-scoped row added to `test/it/postgres/seed-data/feature-flags.js` (inserted `asWriter`, since `feature_flags` grants INSERT to `postgrest_writer` only).
- [ ] project-elmo-ui: select the Semrush create path from that signal rather than from `serenity_ui` / the LD entitlement (separate issue in that repo once the contract is agreed).
- [ ] Backfill: any brand created in Adobe Corp between wave 1 and this change stays flat; decide per brand whether a later wave migrates it or ops provisions it.

## Open decisions

- **Signal derivation.** Derive it from migration state as proposed above, or introduce the explicit `serenity_new_brands` organization flag, or derive it with an explicit `false` acting as an off-switch for a paused migration.
- **Where the signal surfaces.** A new derived boolean on the organization payload, or an entry on `GET /organizations/:organizationId/feature-flags/llmo`, which project-elmo-ui already fetches for `serenity_ui` and which would need no new contract — at the cost of putting derived state in a listing of table rows.
- **Stamp-failure semantics.** Soft-delete the brand and empty the workspace, accepting that a `status = 'deleted'` row remains; hard-delete the brand row and its children so nothing remains; or treat the stamp as best-effort, returning 201 and leaving a provisioned brand that reads as classic until ops or a later wave heals it.

## Preconditions outside this change

Both verified today, and both bite the moment self-serve creation is live for Adobe:

- **Model set.** `provisionBrandSubworkspace` honours a caller-supplied non-empty `semrushModelIds` list as-is, and falls back to the canonical net-new default only when the caller supplies none (`src/support/serenity/brand-provisioning.js:175`). That default is 8 models (`NET_NEW_DEFAULT_MODEL_KEYS`, `src/support/serenity/default-models.js:35`: `search-gpt`, `google-ai-overview`, `google-ai-mode`, `gemini-2.5-flash`, `claude-sonnet-4`, `perplexity`, `grok-3`, `deepseek`). Adobe's contract is four — ChatGPT, Gemini, Google AI Mode, Google AI Overview — which is what wave 1 provisioned. So a wizard create that sends the four is already correct, while the sub-workspace-only create and pending activation, which send none, land on the eight. Honouring a recorded per-organization model set is the open item on https://github.com/adobe/spacecat-api-service/issues/3020 and closes that gap.
- **Allocation.** A child workspace is created with an empty `resources` object (`src/support/serenity/rest-transport.js:1031`), so it starts with zero AI allocation, and the just-in-time allocator is off in prod (`SERENITY_DYNAMIC_ALLOCATION = false` in `dx_mysticat/prod/api-service`, read 2026-08-13; `src/support/serenity/dynamic-allocation-active.js:56`). Brand creation does thread the allocator through (`brand-provisioning.js:181`), so enabling it for this tenant is sufficient — the per-brand ceilings are already set at 3 projects / 200 prompts. Wave 1's three children had their allocation granted by the Semrush partner out of band; a self-serve brand has no such grant, so without either the allocator or a partner-pre-granted pool its first metered write has nothing to draw on.

## Validation gates

- In dev, against an organization with one brand-scoped `serenity = true` row and no organization row: a create provisions a sub-workspace, writes the brand's `true` override, and the brand answers 200 on the elements, prompts and `/serenity/*` surfaces — including on the request immediately following the create, so the cache invalidation is exercised rather than the TTL.
- The 201 body of that create reports `serenityActive: true`.
- A sibling brand of the same organization with no override is unaffected: still classic, still editable in the legacy prompt surfaces.
- A brand written by `llmo-onboarding.js` in the same organization stays flat and unstamped.
- A provisioning failure leaves no brand row and no live sub-workspace.
- A stamp failure resolves to whichever semantics are chosen above, and the resulting state is asserted explicitly rather than left implicit.
- A subsequent migration-CLI `plan` for a later wave excludes the Semrush-native brand (it carries a sub-workspace and a `true` override, so it is neither a migration candidate nor a fence candidate).

## References

- Per-brand rollout resolution and the `feature_flags.brand_id` override model: https://github.com/adobe/spacecat-api-service/issues/3020 and https://github.com/adobe/spacecat-api-service/pull/3024
- Schema and composite FK: https://github.com/adobe/mysticat-data-service/pull/891
- Per-brand wave model and Adobe wave 1: https://github.com/adobe/serenity-docs/issues/317 , https://github.com/adobe/serenity-docs/pull/336 , https://github.com/adobe/serenity-docs/pull/337