camunda / camunda/api-test-generator

Set a baseline for OCA tests

Open
#540 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
0
Forks
3
Avg merge
13h 41m
Merged PRs (30d)
23

Description

Parent: #538
Blocked by: #539

The 280-failure report in the parent cannot be reproduced. The container that produced it is gone.

It ran from docker/docker-compose.yml with CAMUNDA_DATA_SECONDARY_STORAGE_TYPE: rdbms,
CAMUNDA_SECURITY_AUTHORIZATIONS_ENABLED: false and UNPROTECTEDAPI: true, on image tag
8.9-SNAPSHOT that self-reported 8.10.0-alpha4. Something replaced it mid-investigation with
camunda/camunda:SNAPSHOT on Elasticsearch, requiring auth. The two disagree on at least 47 results,
including whether four route families exist at all. The parent has the comparison table.

Until a run exists that anyone can repeat, no failure count means anything.

What to do

Run the on-demand workflow from #539 against a clean container and record the result. Attach the breakdown to this issue so later PRs have something to diff against.

The group D experiment, done here

Group D is 47 failures from routes that returned No static resource on the report's deployment but
respond on Elasticsearch. Try the compose correction as part of this run: switch
CAMUNDA_DATA_SECONDARY_STORAGE_TYPE off rdbms, add Elasticsearch, and set
CAMUNDA_SECURITY_AUTHORIZATIONS_ENABLED: true.

Watch for one side effect before committing to it. The unsecured profile depends on
UNPROTECTEDAPI: true with authorizations off, which is how its requests reach the API with no
credentials. Every unauthenticated call I made against the Elasticsearch container returned 401. So
turning authorizations on may change what "unsecured" means for the whole suite, and could trade 47
failures for a much larger auth-shaped set.

Measure it. If the corrected compose clears group D without wrecking the rest, take it and close
group D here. If it trades one problem for a bigger one, then the team is genuinely running two
deployment shapes, and the answer is a deployment dimension in the config layer: a scoped block in
request-validation.json, a validator beside config.ts:227-249, selection via an env var alongside
RV_PROFILE, threading through generate.ts:193-204, and guardrail tests. Open that as a separate
issue if it comes to it.

While you are here, re-check group J. Two backup operations return 500 with
(code: UNSUPPORTED_MESSAGE), which looks like the same rdbms limitation. If they disappear on
Elasticsearch, fold them into group D and drop the planned upstream filing.

Acceptance

  • A recorded baseline run from a checked-in compose file, with the breakdown attached here.
  • A decision on group D, with the numbers behind it.
  • Group J confirmed as either environmental or a genuine server bug.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the on-demand workflow from #539 and the checked-in docker/docker-compose.yml, using the configuration values and image details in this issue. Record the clean run and compare its breakdown with the parent’s groups, especially D and J. Done means an attached baseline, a measured group D decision, and confirmation of whether group J is environmental or a server bug.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker-compose, elasticsearch, typescript
Domain
devops, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.