adobe / adobe/spacecat-shared

project-engine-client mock: benchmark batch-create wrongly 409s on a duplicate domain

Open
#1,888 0 comments 0 reactions 1 assignee Claimed by @aliciadriani View on GitHub
Dominant language
JavaScript
Stars
5
Forks
2
Avg merge
19h 55m
Merged PRs (30d)
33

Description

## Summary

The stateful mock for `POST /v2/workspaces/{id}/projects/{project_id}/ai_models/benchmarks` treats a benchmark's `domain` as a conflict token, so it rejects a batch that the live Project Engine API accepts. A domain is not unique within an AIO project — benchmarks are discriminated by `brand_name` — and the mock is the only thing that says otherwise.

## Where

`packages/spacecat-shared-project-engine-client/mock/counterfact/routes/v2/workspaces/{id}/projects/{project_id}/ai_models/benchmarks.js`

- Line 29 — `b?.domain` is folded into `identityTokens`, alongside `brand_name` and `brand_aliases`.
- Lines 21-22 and 41-42 — the docstring and the inline comment both assert that a colliding domain is a hard 409, citing a live verification on 2026-06-29.

The v1 list/read routes and the `{benchmark_id}` PUT handler do not perform this check, so the defect is confined to the v2 batch-create handler.

## Evidence that the live API disagrees

Probed against the live Project Engine on 2026-08-19 (dev workspace, `https://adobe-hackathon.semrush.com`), three cases:

- Creating a benchmark on a domain that an existing benchmark already uses — **accepted**.
- A single batch containing two benchmarks that share one domain and differ by `brand_name` — **accepted**.
- Control: a batch whose `brand_name` duplicates an existing benchmark's — **rejected with 409**.

Created rows were deleted afterwards and the project verified back to zero.

This matches production data: a read of a prod project on 2026-08-18 found five benchmarks sharing `nba.com`, distinguished only by `brand_name`.

The handler is also internally inconsistent — the 409 body it returns on line 56 reads `ai benchmark conflict: duplicate brand name or alias` and never mentions the domain it just rejected on.

## Impact

A consumer cannot integration-test any write that legitimately puts two benchmarks on one host. This surfaced while fixing the competitor-benchmark sync in spacecat-api-service, which was keying on domain for the same wrong reason:

https://github.com/adobe/spacecat-api-service/pull/3101

That PR re-keys the sync onto `brand_name`. Its integration coverage had to be trimmed to assertions the mock can express, and the sibling-benchmark behaviour is proven only by unit tests plus the live probe above — the one path where the mock actively blocks the test that would be most valuable.

## Suggested fix

- Drop `b?.domain` from `identityTokens` so conflicts key on `brand_name` and `brand_aliases` only, matching the 409 message the handler already returns.
- Correct the docstring and the inline comment, replacing the 2026-06-29 claim with the current contract.
- Add a case covering two benchmarks that share a domain and differ by name, asserting the batch succeeds.

Related mock-fidelity fix: https://github.com/adobe/spacecat-shared/issues/1856

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.