Committed generated/ and stubs/ drift from what CI regenerates, undetected

Problem

The committed generated/ and stubs/ artifacts on main are stale relative to what CI actually builds, and nothing detects it.

CI regenerates from scratch on every run:

• Clear checked-in spec (prevent stale fallback) → rm -rf external-spec/bundled/*
• Download bundled spec → fresh bundle from camunda/camunda@main
• Generate SDK → make generate-only

It then type-checks, tests and coverage-checks that freshly generated tree. At no point does it compare the regenerated output against the generated/ and stubs/ trees committed in the repo. As a result the two can diverge without bound, and they have.

How this bit us

While fixing red CI in #267, examples/agent_instance.py looked correct against the local checkout — the committed stubs/ described the old agent-instance API shape. CI disagreed, because CI had regenerated against current upstream where the shape had changed:

• AgentInstanceCreationRequest lost definition and gained required element_instance_key, job_key, job_lease, history
• AgentInstanceUpdateRequest gained required job_key and job_lease
• the createAgentInstanceHistoryItem operation was removed entirely

The stale stubs actively misled the diagnosis. The only way to get ground truth was to run a full local regeneration and read the generated classes directly.

This is not specific to agent instances — it is a property of the setup. Any upstream schema change produces the same trap, and the cost lands on whoever is next debugging a failure in that area.

Why it also weakens review

Because the committed artifacts are never validated, a PR can hand-edit generated/ or stubs/ — or simply commit output from a differently-configured local run — and CI will happily pass, since it discards those files and regenerates. The committed tree is presented to readers as the generated output, but nothing guarantees it is.

Proposed fix

Add a generation drift job to ci.yml, mirroring the pattern already used in the Go and Rust SDKs:

1. bundle the spec (reuse the existing bundle-spec artifact)
2. make generate-only
3. git diff --exit-code -- generated stubs

On drift, fail with the diff in the log — or, to avoid making routine upstream churn block unrelated PRs, upload the diff as a patch artifact and emit a warning (the Go SDK's report-only Generation drift job is the reference implementation).

Two details worth getting right:

• Do not use a bare git diff to decide. Regeneration can add untracked files, which git diff does not see. Use git status --porcelain -- generated stubs or git add -A before diffing. (This same blindness exists today in the Go/C#/JS drift jobs and should be fixed there too.)
• Decide explicitly whether the committed artifacts are source of truth or convenience only. If convenience only, the cheaper fix is to stop committing them and let consumers generate — but that breaks IDE navigation for anyone reading the repo, so a drift check is likely the better trade.

Acceptance criteria

• CI detects when generated/ or stubs/ on a branch differ from a clean make generate-only against the bundled spec.
• The check accounts for added and removed files, not just modified ones.
• A regeneration PR that brings the committed artifacts back in line passes cleanly.
• The behaviour on upstream-churn drift (fail vs. warn + patch artifact) is a deliberate, documented choice.

Start with ci.yml and the existing bundle-spec artifact, then trace the make generate-only flow and the Go and Rust SDK drift jobs mentioned in the issue. Add a generation-drift check covering generated/ and stubs/, including added and removed files, and document whether upstream drift fails CI or produces a warning and patch artifact. Verify that a clean regeneration passes and intentional drift is detected.