camunda / camunda/api-test-generator
Promote serverEmergent keys to runtimeEmission
@jwulf is already working on this.
Since May 20, 2026.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 3
- Avg merge
- 13h 41m
- Merged PRs (30d)
- 23
Description
Problem
The planner cannot generate a meaningful test for any mutation operation whose key is produced as a side-effect of process execution (updateUserTask, completeUserTask, assignUserTask, updateJob, failJob, resolveIncident, correlateMessage, …). It falls through to a synthetic seedBinding(...) placeholder, the PATCH/POST is dispatched against a key that doesn't exist, and the realistic outcome is 404/400 — not a real test of the endpoint.
Today this is hidden because every such operation is currently classified kind: "serverEmergent" in configs/camunda-oca/ontology/semantics.json with the comment "no direct create API → planner mints a deterministic placeholder". That classification is the surrender, not a description: the keys are reachable, the planner just isn't allowed to reach them.
A worked example: updateUserTask against the bundled spec emits
ctx.userTaskKeyVar = ctx.userTaskKeyVar ?? seedBinding('userTaskKeyVar');
const url = `${baseUrl}/user-tasks/${ctx.userTaskKeyVar || '${userTaskKey}'}`;
const resp1 = await request.patch(url, { headers: await authHeaders(), data: {} });
expect(resp1.status()).toBe(204);
— a synthetic key, an empty body, and a status assertion that cannot succeed on a real cluster.
Root cause (two intersecting defects)
1. Deployment escaped the prereq chain via the codegen role hook
configs/camunda-oca/codegen/playwright/roles/deploymentGateway/ deploys BPMN fixtures before the Playwright suite runs. That structure was an emitter ergonomic (so every test doesn't re-deploy the same BPMN), not a semantic replacement for declaring deployment as a planner-visible prerequisite. The planner therefore has no notion that createProcessInstance requires ModelDeployed, and no notion that some BPMN fixtures provide ModelHasUserTask / ModelHasServiceTask / etc.
The fix is to put deployment back into the prereq graph as a real producer step. The role hook can remain as an optimisation (e.g. dedupe-by-fixture-hash at emission time), but the planner has to see it.
2. runtimeEmission is not modelled
Several keys currently bucketed as serverEmergent are in fact discoverable: they appear on a known search endpoint, under eventual consistency, after a specific upstream side-effect (process execution, job activation, incident raising). They differ from true serverEmergent keys (e.g. internally-generated correlation IDs that no API surface returns) and they need a different planning strategy.
Proposed new semantic-type kind:
"UserTaskKey": {
"kind": "runtimeEmission",
"intersectsWith": "serverEmergent",
"emittedBy": {
"predecessor": "ProcessInstanceExists",
"guardedBy": ["ModelHasUserTask"]
},
"discoveredVia": {
"operationId": "searchUserTasks",
"filterBy": "processInstanceKey",
"extractKey": "userTaskKey",
"consistency": "eventual"
}
}
runtimeEmission intersects with serverEmergent — both describe keys with no direct create endpoint. The distinction is that runtimeEmission carries the planner information needed to reach the key: producing predecessor + discovery operation + consistency model. A pure-serverEmergent key (nothing discoverable) still falls through to placeholder, which is the correct outcome for that subset.
This needs:
- A new edge kind in
path-analyser/src/ontology/edgeSchema.ts(or a sibling vocabulary) for runtime emission — distinct from the existing membership edges (PUT/DELETE between two extant entity kinds). Membership edges describe what the API offers; runtime-emission edges describe what the system produces during execution. - Capability declarations in
configs/camunda-oca/ontology/semantics.json(ModelHasUserTask,ModelHasServiceTask,ModelHasJob,ModelEmitsIncident, …) and corresponding fixture-metadata entries inconfigs/camunda-oca/fixtures/deployment-artifacts.jsonso the planner can pick the right BPMN for the capability the test needs. - Planner support in
path-analyser/src/scenarioGenerator.ts: when a missing prereq is aruntimeEmissionkey, plan
createDeployment(fixture-with-capability) → createProcessInstance(deployed) → poll search-with-eventual-consistency → bind key → operation under test. - The
await-eventuallyhelper atmaterializer/src/playwright/support/await-eventually.tsis already there; the planner just needs to wire it in.
Read-back after mutate — belongs in scenario-template vocabulary
The current path-analyser/src/ontology/scenarioTemplateSchema.ts encodes test patterns as PrereqChain / Invoke / Observe steps applied to Edge and Entity subjects. "Mutate then re-fetch and assert the field changed" is a fourth pattern with the same shape and belongs there, not in the planner's hard-coded chain logic. Sketch:
{
"name": "UpdatedFieldVisibleOnReadBack",
"appliesTo": { "kind": "RuntimeEntity" }, // or extend Entity
"steps": [
{ "kind": "PrereqChain", "for": "mutator" },
{ "kind": "Invoke", "op": "mutator" },
{ "kind": "Observe",
"op": "fetcher",
"expect": "fieldEquals",
"field": "<resolved-from-mutator-request-body>" }
]
}
This generalises beyond user tasks: every updateXxx / assignXxx / unassignXxx becomes a candidate for an automatic read-back-after-mutate scenario, without each one being baked into the planner.
Class scope
The defect applies to every serverEmergent-classified key that is in fact reachable via search:
UserTaskKey→searchUserTasks(processInstanceKey)JobKey→searchJobs(processInstanceKey)IncidentKey→searchIncidents(processInstanceKey)MessageSubscriptionKey→searchMessageSubscriptions(processInstanceKey)VariableKey→searchVariables(scopeKey)DeploymentKey→ returned synchronously fromcreateDeployment(this one is closer to a missing producer wiring than a runtime-emission case, but it's in the same bucket today)ProcessInstanceKeyitself — already a producer, but aruntimeEmissionview would make the dependency onModelDeployedexplicit instead of relying on the role hook
A targeted regression invariant: every operation that consumes a runtimeEmission key in a path-param either has a planner-derived discovery chain leading to that key, or the scenario is explicitly suppressed.
Acceptance criteria
- New ontology vocabulary:
runtimeEmissionsemantic-type kind (intersecting withserverEmergent) withemittedBypredecessor + capability guards anddiscoveredViasearch-op + consistency model. - Capability + fixture metadata for BPMN fixtures (
ModelHasUserTaskfirst; others to follow). - Deployment surfaced as a planner-visible producer of
ModelDeployed+ capabilities; the codegendeploymentGatewayrole degrades to an emission-time dedupe optimisation, not a semantic substitute. scenarioGenerator.tsplans thecreateDeployment → createProcessInstance → search-with-eventual-consistency → bind key → operation under testchain when the missing prereq is aruntimeEmissionkey.Observestep extended (or sibling step added) in scenario-template vocabulary to cover "read-back-after-mutate, assert field equals submitted value".- A class-scoped L3 invariant under
configs/camunda-oca/regression-invariants.test.ts: no operation that consumes aruntimeEmissionkey in a path-param compiles into a scenario whose only producer for that key isseedBinding(...)against a string placeholder.
Out of scope
- Pure
serverEmergentkeys with no discovery surface (those continue to fall through to placeholders). - Negative tests for
updateXxxon a non-existent key — separately useful, owned byrequest-validation. - Idempotency / cleanup of deployed BPMN between runs — separate concern, intersecting with #304 (uniqueness of test identifiers).
Notes
- This is the root cause for the long tail of
updateXxx/assignXxx/completeUserTask/ etc. tests being effectively no-ops. - The fix is principled rather than mechanical: it gives the planner the same view the human author of an integration test has — "to update a thing, I need to make a thing, and to make a thing of this kind I need this BPMN to be deployed".
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.