camunda / camunda/api-test-generator

awaitEventually witnesses time out for 10 search/get operations against live broker

Open
#176 1 comment 0 reactions 1 assignee View on GitHub

@jwulf is already working on this.

Since May 13, 2026.

OCA
Dominant language
TypeScript
Stars
0
Forks
3
Avg merge
13h 41m
Merged PRs (30d)
23

Description

Observation

When the positive suite is executed against a live broker, 51 scenarios fail with EventualConsistencyTimeoutError from awaitEventually(...) (10s default). The witness condition never observes the expected state within the deadline.

Breakdown by witness operation:

  • searchAuditLogs (16)
  • searchCorrelatedMessageSubscriptions (7)
  • searchVariables, searchUserTasks, searchGlobalTaskListeners, searchBatchOperations (5 each)
  • searchMessageSubscriptions, searchBatchOperationItems (4 each)
  • getAuditLog, getAgentInstance (1 each)

All witness operations return lastStatus=200 — the read succeeds, but the body never contains the expected state. Most attempts hit ~21 polls in ~10s.

Hypothesis (needs confirmation)

Three plausible root causes; the relative weight is unclear without further investigation:

  1. Search witness queries are missing a filter so they return an empty page even though the entity exists. The awaitEventually predicate sees no rows and times out.
  2. Search-index lag exceeds the 10s deadline under the H2-in-memory configuration in docker/docker-compose.yml (ZEEBE_BROKER_EXPORTERS_RDBMS_ARGS_FLUSH_INTERVAL: PT0S, but the secondary-storage indexer still has its own pipeline).
  3. The witness operation observes a downstream projection that never materialises because an upstream step (the actual mutation) silently failed earlier in the chain. The suite's expect(resp.status()).toBe(...) already pinned the upstream, so this would specifically be a case where the chain semantics expect a side effect that the broker doesn't produce.

Reproducer

docker compose -f docker/docker-compose.yml up -d
TEST_SEED=snapshot-baseline npm run testsuite:generate
cd generated/camunda-oca/playwright && npm install
API_BASE_URL=http://localhost:8080/v2 npx playwright test --grep "searchAuditLogs"

Suggested direction

For each witness operation in the list above:

  1. Manually run the chain up to the witness call and capture the actual response body.
  2. Compare to the witness predicate emitted in the spec.
  3. Decide per-operation: tighten the filter, raise the deadline, or report upstream that the side effect isn't being projected.

The L3 invariants (every eventually-consistent read step is wrapped with awaitEventually (#106)) prove the wrap is correctly emitted; this issue is about whether the wrap is productive.

Out of scope

Removing the deadline. The cap is required to keep the suite bounded.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.