camunda / camunda/orchestration-cluster-api-python
Nullable-field tolerance covers 5 fields in 1 model; 620 bare d.pop() calls can KeyError
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 6h 15m
- Merged PRs (30d)
- 20
Description
Problem
hooks/post_gen/0450_tolerant_nullable_pop.py makes d.pop() tolerant of missing keys, but only for five hardcoded fields in one hardcoded file:
target = out_dir / "camunda_orchestration_sdk" / "models" / "deployment_metadata_result.py"
_NULLABLE_FIELDS = {
"processDefinition", "decisionDefinition", "decisionRequirements", "form", "resource",
}
Every other nullable field in every other model keeps a bare d.pop("fieldName"), which raises KeyError when a server omits the key. Servers legitimately omit null-valued keys, and released servers omit fields the spec has since added.
Scope
Measured against the currently bundled spec and the generated models on main:
| count | |
|---|---|
fields the spec marks nullable |
138 |
bare d.pop() calls in generated models |
1304 |
| of those, on fields the spec marks nullable | 620, across 190 model files |
| distinct field names at risk | 128 |
| covered by hook 0450 today | 5, in 1 file |
This has already cost us a day
The concrete failure was jobLeaseToken. Upstream renamed it and made it required-but-nullable; ActivatedJobResult.from_dict did d.pop("jobLeaseToken") with no default; the server CI pinned did not send it. The result:
ERROR | camunda_orchestration_sdk.runtime.logging:error:84 - Error polling: 'jobLeaseToken'
The job worker then never handled a job, so create_process_instance(await_completion=True) timed out and surfaced as HTTP 504 DEADLINE_EXCEEDED from the gateway — two layers away from the actual cause, and indistinguishable from broker trouble. 74 of 76 integration tests passed throughout, so it read as an infrastructure blip rather than a deserialization bug.
#286/#284 removed that particular trigger by keeping the spec and the server image in step. They did not fix this: the next nullable field a server omits raises the same KeyError, with the same misleading symptom.
Suggested fix
Derive tolerance from the spec instead of a list. For every field the spec marks nullable, emit d.pop("field", None):
- the hook already has the bundled spec available in
context - "absent" and "present but null" are the same thing for a nullable field, so defaulting to
Noneis semantically correct rather than merely lenient - it removes the per-field maintenance entirely — nothing to keep in step with a rename
Worth pairing with a test that asserts the class: no generated model contains a bare d.pop() for a field the spec marks nullable.
Why this matters beyond Python
The same hardcoded-list pattern rotted in four places during one upstream rename:
| repo | mechanism | outcome |
|---|---|---|
| Rust | hook_08 _DEFAULTABLE matched a literal serde attribute |
silently stopped applying; no test caught it |
| Go | hook_05 VERSION_SKEW_OPTIONAL listed the old wire name |
relaxed 18 → 17 fields; its test caught it |
| Go | hook_01 _EXTRA_SCALAR_TYPES hardcoded the type |
duplicate type; build failed |
| Python | 0450 hardcoded file + field list |
this issue |
Go's is the useful counter-example: same rot, but TestActivatedJobDecodesWithMissingVersionSkewFields named the field immediately. The failure mode of a hardcoded list is silence, so the fix is either to stop hardcoding (preferred here, since the spec already carries the information) or to fail loudly when an entry stops matching.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with hooks/post_gen/0450_tolerant_nullable_pop.py and inspect how the bundled spec is exposed through context. Compare its handling with bare d.pop() calls in the generated model files, then add a test that checks nullable spec fields are not emitted as bare pops. Done means nullable fields tolerate omission across the generated models without maintaining a hardcoded field list.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100