camunda / camunda/api-test-generator

Meta: encode test-meaningful behaviour for runtime entities — supersedes #150

Open
#192 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

design discussion layer-0 meta
Dominant language
TypeScript
Stars
0
Forks
3
Avg merge
13h 41m
Merged PRs (30d)
23

Description

Supersedes #150. The discussion in #150 evolved from a two-option choice (directlyExposedBy vs runtimeEntities) into a four-layer model and a four-class taxonomy of test-meaningful behaviour. This meta-issue tracks the resulting workstream; the original design discussion is preserved in #150's comments.

Background — what we converged on

The variant planner today emits tests that check little more than HTTP status. To emit meaningful tests it needs three things the OpenAPI spec does not encode:

  1. Which runtime entity sits behind a semantic identifier (ElementInstanceKey, IncidentKey, JobKey, UserTaskKey).
  2. Which lifecycle states the entity can be in, and which operations drive transitions.
  3. Which model-level elements in a deployed artefact (BPMN file) are valid targets for a given operation, and what effect targeting them has on the running system.

These split into four layers (see #150 comments for the full derivation):

Layer Encodes Status
0. Model semantics per-fixture BPMN element metadata: type, validStartTarget, semanticRole, expectedNextSteps new
1. Artefact-derived raw values extract values from artefact bytes (e.g. element ids from BPMN) new
2. Direct runtime exposers which ops authoritatively expose a runtime identifier (directlyExposedBy or runtimeEntities + boundTo) new — was the original A/B question
3. Indirect re-projectors today's positional fallback unchanged

Layer 2 alone fixes only ~2 of the 12 ElementInstanceKey-consumer use-cases in #150's taxonomy (visualisation, audit). The other 10 (modification, variable scoping, user-task interaction, decisions, message correlation, jobs, ad-hoc, batch) need layer 0.

The four test classes

Independently, the discussion identified that test-meaningful behaviour breaks into four (pre-state, op, post-state, verifier) classes. Each gets its own child issue:

Class Shape Driver issue
1 — Existence transition (CRUD) entity ∉ store → entity ∈ store (or vice versa) #188
2 — State-machine transition on a runtime entity entity in state S1 → entity in state S2 #189
3 — Side-effect emission downstream entity appears in another store #190
4 — Aggregate invariant aggregate or projection moves in a known direction #191

Class 2 (#189) is the layer-0 driver: encoding an entity's lifecycle and how ops drive it through that lifecycle is what unlocks modifyProcessInstance, cancelProcessInstance, resolveIncident, completeJob, completeUserTask, etc. Class 3 (#190) reuses Class 2's vocabulary for a downstream-entity verifier. Class 1 (#188) is largely already modelled. Class 4 (#191) is a polish layer.

Concrete work items

  • #188 — Class 1: existence transitions (CRUD) — gap analysis
  • #189 — Class 2: state-machine transitions on runtime entities — primary layer-0 driver
  • #190 — Class 3: side-effect emission tests
  • #191 — Class 4: aggregate-invariant tests
  • Layer-2 ranking PR (directlyExposedBy or equivalent) — small fix to unblock #139 chain shape; non-foreclosing of layer 0
  • Decide upstream-spec ask: should DeploymentProcessResult surface elementIds[] as provider:true? (See #150 comment "Tightened framing".)

Out of scope for this meta

  • Any change to the provider:true semantics of existing leaves.
  • A "manifest diff" style end-to-end snapshot guard (retired deliberately — see README).

Acceptance criteria for closing this meta

  1. All four child issues either closed or explicitly marked won't-do with a recorded reason.
  2. The two currently-skipped #37 invariants in configs/camunda-oca/regression-invariants.test.ts re-enabled and passing.
  3. createProcessInstance.startInstructions[].elementId produces a meaningful chain (not the throwaway-process warm-up of #139).
  4. At least one Class 2 test (e.g. modifyProcessInstance terminate) is generated end-to-end with an act+verify shape and runs green against the live broker.

cc/ #139 (the concrete planner regression that motivated #150).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the four child issues (#188–#191) and the acceptance criteria for this meta-issue. Inspect configs/camunda-oca/regression-invariants.test.ts and the existing README context, then determine how the child work can re-enable the two skipped invariants and produce a meaningful createProcessInstance chain plus one end-to-end Class 2 test.

Written by the indexing model from the issue text.

Assessment

Tech stack
openapi, typescript
Domain
api, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.