camunda / camunda/orchestration-cluster-api-python

Nullable-field tolerance covers 5 fields in 1 model; 620 bare d.pop() calls can KeyError

Open
#287 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
0
Forks
0
Avg merge
6h 15m
Merged PRs (30d)
20

Description

Problem

hooks/post_gen/0450_tolerant_nullable_pop.py makes d.pop() tolerant of missing keys, but only for five hardcoded fields in one hardcoded file:

target = out_dir / "camunda_orchestration_sdk" / "models" / "deployment_metadata_result.py"

_NULLABLE_FIELDS = {
    "processDefinition", "decisionDefinition", "decisionRequirements", "form", "resource",
}

Every other nullable field in every other model keeps a bare d.pop("fieldName"), which raises KeyError when a server omits the key. Servers legitimately omit null-valued keys, and released servers omit fields the spec has since added.

Scope

Measured against the currently bundled spec and the generated models on main:

count
fields the spec marks nullable 138
bare d.pop() calls in generated models 1304
of those, on fields the spec marks nullable 620, across 190 model files
distinct field names at risk 128
covered by hook 0450 today 5, in 1 file

This has already cost us a day

The concrete failure was jobLeaseToken. Upstream renamed it and made it required-but-nullable; ActivatedJobResult.from_dict did d.pop("jobLeaseToken") with no default; the server CI pinned did not send it. The result:

ERROR | camunda_orchestration_sdk.runtime.logging:error:84 - Error polling: 'jobLeaseToken'

The job worker then never handled a job, so create_process_instance(await_completion=True) timed out and surfaced as HTTP 504 DEADLINE_EXCEEDED from the gateway — two layers away from the actual cause, and indistinguishable from broker trouble. 74 of 76 integration tests passed throughout, so it read as an infrastructure blip rather than a deserialization bug.

#286/#284 removed that particular trigger by keeping the spec and the server image in step. They did not fix this: the next nullable field a server omits raises the same KeyError, with the same misleading symptom.

Suggested fix

Derive tolerance from the spec instead of a list. For every field the spec marks nullable, emit d.pop("field", None):

  • the hook already has the bundled spec available in context
  • "absent" and "present but null" are the same thing for a nullable field, so defaulting to None is semantically correct rather than merely lenient
  • it removes the per-field maintenance entirely — nothing to keep in step with a rename

Worth pairing with a test that asserts the class: no generated model contains a bare d.pop() for a field the spec marks nullable.

Why this matters beyond Python

The same hardcoded-list pattern rotted in four places during one upstream rename:

repo mechanism outcome
Rust hook_08 _DEFAULTABLE matched a literal serde attribute silently stopped applying; no test caught it
Go hook_05 VERSION_SKEW_OPTIONAL listed the old wire name relaxed 18 → 17 fields; its test caught it
Go hook_01 _EXTRA_SCALAR_TYPES hardcoded the type duplicate type; build failed
Python 0450 hardcoded file + field list this issue

Go's is the useful counter-example: same rot, but TestActivatedJobDecodesWithMissingVersionSkewFields named the field immediately. The failure mode of a hardcoded list is silence, so the fix is either to stop hardcoding (preferred here, since the spec already carries the information) or to fail loudly when an entry stops matching.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with hooks/post_gen/0450_tolerant_nullable_pop.py and inspect how the bundled spec is exposed through context. Compare its handling with bare d.pop() calls in the generated model files, then add a test that checks nullable spec fields are not emitted as bare pops. Done means nullable fields tolerate omission across the generated models without maintaining a hardcoded field list.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.