camunda / camunda/api-test-generator

Generator seeds object-typed required fields with placeholder strings — 'Request property [filter] cannot be parsed'

Open
#173 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

OCA
Dominant language
TypeScript
Stars
0
Forks
3
Avg merge
13h 41m
Merged PRs (30d)
23

Description

Observation

When the positive suite is run against a live broker, 53 scenarios fail with:

```
400 Bad Request
Request property [filter] cannot be parsed
```

The generator emits e.g.:

```ts
if (ctx.filterVar === undefined) {
ctx.filterVar = seedBinding('filterVar'); // returns a plain string
}
// ...
const body1 = { filter: ctx.filterVar };
await request.post(`${baseUrl}/process-instances/cancellation`, { data: body1 });
```

But the OpenAPI schema for these batch-operation endpoints declares `filter` as an object (e.g. `{ processInstanceKey: { $eq: ... } }`), not a string. `seedBinding` falls back to a string-shaped placeholder because `seed-rules.json` has no rule matching `filterVar` / object schemas.

Reproducer

```
TEST_SEED=snapshot-baseline npm run testsuite:generate
API_BASE_URL=http://localhost:8080/v2 npx playwright test cancelProcessInstancesBatchOperation.feature.spec.ts
```

All 4 scenarios fail with the body above.

Defect class

The same root cause appears in other request-property-parse failures observed in the same run:

Property Count Notes
`filter` 53 batch-op filters (object)
`processDefinitionId` 32 search-payload bodies (string but with format constraint)
`elementInstanceKey` 27 int64 stringified — generator emits `elementInstanceKey_xxxx`
`processDefinitionKey`, `processInstanceKey`, `decisionDefinitionKey`, `decisionRequirementsKey`, `decisionEvaluationKey`, `jobKey`, `messageSubscriptionKey`, `userTaskKey`, `variableKey`, `auditLogKey`, `deploymentKey`, `formKey`, `scopeKey` 60+ int64 keys submitted with non-numeric placeholders
`variables` 5 object schema
`changeset`, `after`, `from`, `timestamp` 11 typed fields

Two sub-classes:

  1. Object-typed body fields seeded as strings (`filter`, `variables`, `changeset`). The body fails JSON binding outright.
  2. String-typed but format-constrained int64 keys submitted with the literal placeholder name as a value (`processDefinitionKey: 'processDefinitionKey_8py2'`). The broker accepts the JSON but the binder rejects the cast to long.

Suggested direction

  1. Extend `seed-rules.json` (or the seeding mechanism) so:
    • Properties named `filter` / `changeset` / `variables` resolve to a schema-aware empty object `{}` (or a minimal valid filter clause).
    • Properties whose JSON schema is `string` with `pattern: '^[0-9]+$'` (or `format: int64`) emit a numeric string seed like `'2251799813685000'`.
  2. Alternatively: when no semantic provider is available for an int64-key consumer, fall back to omitting the optional field rather than emitting a placeholder that's guaranteed to fail the broker's binder.

Why this hides real bugs

These tests pass HTTP-layer validation but never reach the business-logic layer. A real defect in (say) batch-op filter handling would never be exercised by the current suite.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with seed-rules.json and the seedBinding mechanism, then run TEST_SEED=snapshot-baseline npm run testsuite:generate and the provided Playwright command against the live broker. Trace how object fields and int64-constrained string fields are seeded. Done means the affected request properties receive schema-valid values or are omitted when appropriate, and the reproduced scenarios no longer fail during request binding.

Written by the indexing model from the issue text.

Assessment

Tech stack
openapi, playwright, typescript
Domain
backend-api-design, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.