[ENHANCEMENT](schema) Adopt a policy: schema fields carry no non-null default

Type

Task

Scope

Multi-theme or Platform

Skillset

engineering

Description

The policy

An Overture schema field carries no non-null default value. A field is either required, or it is optional and has no value when absent. Where a value would otherwise be supplied by a default, it is supplied by the publisher, in the published data.

default is not one concept. Each representation reads it differently, and the two that Overture consumers actually use cannot express it at all.

Where it is read	What default means there
Pydantic, at parse time	Injected at construction. The parsed object carries a value the input never had, and Pydantic does not check it against the field's own constraints unless validate_default=True.
JSON Schema	Annotation only. Validators must not modify the instance; code generators and form builders often do. Same keyword, opposite behavior.
Parquet, the published form	No such concept. A column is nullable or it is not. Absent is null.
SQL over a release (DuckDB, Athena, Spark)	Invisible. WHERE level = 0 misses every feature that relied on the default.

The defaults never reach the published data

Measured against release 2026-08-19.0, one partition per type. Every non-null default in the schema occurs in zero published rows.

Field	Default	Rows measured	Rows equal to the default	Null	Other
Building.level	0	4,778,001	0	4,777,954	47
Water.level	0	1,985,644	0	1,985,620	24
Land.level	0	1,174,629	0	1,174,529	100
SpeedLimitRule.is_max_speed_variable	False	601,764 rules	0	601,497	267 (true)

The only consumer who ever sees level = 0 is one parsing a feature through the Pydantic models -- which is not how Overture data is primarily consumed, notwithstanding that the schema is authored in Pydantic. Everyone reading the release sees level = null.

That the minority path is the affected one makes this worse rather than more tolerable. The default is invisible to the audience the schema exists to serve, and the one path that can see it is handed a value the release does not contain.

Where the default belongs

Publication. If ground level is the right value for a building that does not state one, the pipeline that produces the release writes it, once, where the decision is visible and every consumer sees the same answer.

A column that is overwhelmingly one value is dictionary- and RLE-encoded in Parquet, so populating level explicitly across billions of buildings costs close to nothing on disk.

If instead the right answer is "absent means ground level," that is a sentence in the field description, which every representation carries and every consumer can read.

A default is a claim about what the data says; a description is a claim about what absence means. The schema may make the second and not the first.

What the policy does not forbid

• = None on an optional field. That is absence, not a value.
• The MISSING sentinel (pydantic.experimental.missing_sentinel) on bbox and id. It expresses JSON Schema omitability rather than a data value; see packages/overture-schema-system/src/overture/schema/system/optionality.py.

default_factory is forbidden outright, including factories that produce None. A factory is a Python callable, and where a literal default is at least transportable -- JSON Schema can carry default: 0 even though consumers disagree about what it means -- a callable cannot be rendered into any representation but Python. No schema field uses it today.

Related

• #669 -- connectors = [] is rejected by its own min_length=2; the bug this policy generalizes.
• #692 -- makes segment connectors required; the first removal, already in flight.
• #640 -- support-or-reject decision for the Pydantic Field() kwargs the extractor ignores or crashes on; default is the entry this issue resolves.
• #354 -- where the problem was first named, on operating_status.

Start with packages/overture-schema-system/src/overture/schema/system/optionality.py, then read #640 and the related issues to understand how schema fields and Pydantic Field() arguments are extracted. Done means the no-non-null-default policy, including rejection of default_factory, is consistently enforced while preserving None and MISSING semantics; the issue does not name specific tests or implementation files beyond optionality.py.