camunda / camunda/api-test-generator

Extend agent-assisted fixing to spec-bump-check's broken-drift path

Open
#481 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
0
Forks
3
Avg merge
13h 41m
Merged PRs (30d)
23

Description

Context

PR #472 built a triage agent for the camunda-hub nightly (triage-camunda-hub-nightly.yml) that, within tight guardrails, can apply a minimal fix and open a PR against api-test-generator for two cases: a test-generation bug (generated test itself is wrong) or an unmapped-operation coverage gap. It never touches camunda-hub — issues only there.

spec-bump-check.yml (daily, both configs: camunda-oca + camunda-hub) has an analogous but currently unfixed gap: when latest upstream drifts and the generate/invariants/unmapped-ops check comes back broken (not clean), it only opens a tracking issue — pure reporting, no attempted fix. A human has to manually go model the change (e.g. #386's agent-instance domain work).

Proposal

Extend agent-assisted fixing to spec-bump-check's broken-drift path, reusing the same pattern/guardrails already built in #472:

  • Only attempt a fix when it's minimal, mechanical, and one-sentence-describable (a small suppress-list entry, an obviously-correct small ontology/entity-kind mapping, a narrow assertion fix) — same bar as the existing test-generation/coverage-gap guardrails.
  • Everything else (a whole new domain needing modeling, anything non-trivial) stays report-only, same as today's tracking-issue path — i.e. the agent needs to triage which bucket a given break falls into before attempting anything, not treat every broken-drift failure as fixable.
  • Scoped identically: api-test-generator only, PR-gated, never a direct push to main.

The "clean" bump-PR path (generate/invariants/unmapped-ops all pass) should stay exactly as-is — it's a mechanical re-pin with zero judgment involved; an agent would add cost for zero benefit there.

Related findings from the same discussion (not in scope here, noted for follow-up)

  • camunda-oca has no nightly or triage-agent equivalent at all — no nightly-camunda-oca.yml, nothing like #472 for it. Worth a separate conversation on whether that's intentional (oca is already exercised via ci.yml/spec-bump-check.yml, no live-runtime nightly exists the way hub has one) or a real coverage gap, given oca is a higher-traffic, faster-moving domain than hub.
  • #480 (hub-pr-check reporting green on a PR that introduces an uncovered endpoint) is a separate, mostly-mechanical gap — not agent work, tracked there already.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing spec-bump-check.yml with the triage-camunda-hub-nightly.yml pattern introduced by PR #472. Trace the broken-drift path and define the minimal-fix guardrails, report-only fallback, and PR-gated behavior; done means clean bumps remain unchanged while eligible breaks can produce a guarded api-test-generator PR.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, typescript
Domain
ci-cd, devtools
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.