project-engine-client mock: benchmark batch-create wrongly 409s on a duplicate domain
- Dominant language
- JavaScript
- Stars
- 5
- Forks
- 2
- Avg merge
- 19h 55m
- Merged PRs (30d)
- 33
Description
## Summary
The stateful mock for `POST /v2/workspaces/{id}/projects/{project_id}/ai_models/benchmarks` treats a benchmark's `domain` as a conflict token, so it rejects a batch that the live Project Engine API accepts. A domain is not unique within an AIO project — benchmarks are discriminated by `brand_name` — and the mock is the only thing that says otherwise.
## Where
`packages/spacecat-shared-project-engine-client/mock/counterfact/routes/v2/workspaces/{id}/projects/{project_id}/ai_models/benchmarks.js`
- Line 29 — `b?.domain` is folded into `identityTokens`, alongside `brand_name` and `brand_aliases`.
- Lines 21-22 and 41-42 — the docstring and the inline comment both assert that a colliding domain is a hard 409, citing a live verification on 2026-06-29.
The v1 list/read routes and the `{benchmark_id}` PUT handler do not perform this check, so the defect is confined to the v2 batch-create handler.
## Evidence that the live API disagrees
Probed against the live Project Engine on 2026-08-19 (dev workspace, `https://adobe-hackathon.semrush.com`), three cases:
- Creating a benchmark on a domain that an existing benchmark already uses — **accepted**.
- A single batch containing two benchmarks that share one domain and differ by `brand_name` — **accepted**.
- Control: a batch whose `brand_name` duplicates an existing benchmark's — **rejected with 409**.
Created rows were deleted afterwards and the project verified back to zero.
This matches production data: a read of a prod project on 2026-08-18 found five benchmarks sharing `nba.com`, distinguished only by `brand_name`.
The handler is also internally inconsistent — the 409 body it returns on line 56 reads `ai benchmark conflict: duplicate brand name or alias` and never mentions the domain it just rejected on.
## Impact
A consumer cannot integration-test any write that legitimately puts two benchmarks on one host. This surfaced while fixing the competitor-benchmark sync in spacecat-api-service, which was keying on domain for the same wrong reason:
https://github.com/adobe/spacecat-api-service/pull/3101
That PR re-keys the sync onto `brand_name`. Its integration coverage had to be trimmed to assertions the mock can express, and the sibling-benchmark behaviour is proven only by unit tests plus the live probe above — the one path where the mock actively blocks the test that would be most valuable.
## Suggested fix
- Drop `b?.domain` from `identityTokens` so conflicts key on `brand_name` and `brand_aliases` only, matching the 409 message the handler already returns.
- Correct the docstring and the inline comment, replacing the 2026-06-29 claim with the current contract.
- Add a case covering two benchmarks that share a domain and differ by name, asserting the batch succeeds.
Related mock-fidelity fix: https://github.com/adobe/spacecat-shared/issues/1856
Contributor guide
Assessment
This issue has not been assessed yet.