ome / ome/ngff

expand and constrain axis `unit` values (UDUNITS-2 controlled vocabulary, MUST)

Open
#593 11 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
171
Forks
75
Avg merge
2d 3h
Merged PRs (30d)
20

Description

Summary

This is a request for preliminary feedback, not yet an RFC.

Two parts are proposed:

  1. Expand the enumerated, UDUNITS-2-valid unit vocabularies beyond space and time,
    so that other axis types in the specification (and likely future ones) have canonical units.
  2. Tighten the requirement: unit MAY be omitted, but if present its value MUST be
    one of the strings enumerated for that axis type — rather than the current SHOULD — so
    that it is machine-verifiable in the JSON Schema.

Related: ome/ngff-spec#192, ome/ome-ngff-validator#76, ome/ome-ngff-validator#52

Current state

Since 0.4, an axis "SHOULD contain the field unit [...] The value SHOULD be one of the following
strings, which are valid units according to UDUNITS-2", with enumerations provided only for
space and time axes. In 0.6.dev4, axes moved under coordinateSystems and the recommended
type vocabulary grew to array, space, time, channel, coordinate, displacement, plus
arbitrary custom types, but the unit lists were not revisited and still cover only space and
time.

Because the requirement is SHOULD, and the JSON Schemas encode MUST requirements, the schemas
validate only that unit is a string and never validate its value. That is the gap reported in
ome/ngff-spec#192.

This was already flagged during RFC-5 review. In response to a reviewer question about units, the
authors noted that units were out of scope for RFC-5 and that "revisiting units would be good
topic for a new RFC [...] since they seem not to have been reconsidered since they [were introduced]".
This issue picks up that deferred item.

Why this matters: FAIR interoperability

The value of an enumeration is not the list itself. It is that the same string always denotes the
same quantity, and that any consumer can decide membership without parsing, guessing, or
normalizing.

  • "Valid UDUNITS-2" is a weaker constraint than the enumeration. UDUNITS-2 deliberately supports
    plurals, symbols, prefixes, aliases, and compound expressions. micrometer, micron, microns,
    um, µm (U+00B5 MICRO SIGN), and μm (U+03BC GREEK SMALL LETTER MU) are one physical unit and
    six incompatible strings — the last two are visually identical and byte-different. The enumeration
    is what carries the interoperability guarantee today; the UDUNITS-2 reference supplies provenance
    and conversion semantics.
  • FAIR I1/I2. A controlled vocabulary drawn from an external, well-defined reference gives units
    a formal, shared, machine-actionable representation instead of free text.
  • Cheap, exact, cross-language validation. An enum is validated identically by every JSON
    Schema implementation, in every language, with no unit-parsing dependency. It also surfaces errors
    at write time rather than as silent misinterpretation at read time.
  • Equality and comparison semantics. Canonical strings let readers compare, merge, and register
    datasets by string equality, and let scale factors be converted safely.
Failures already observed
  • ome/ome-ngff-validator#76: a sample in the wild used "units" instead of "unit". It passed
    validation while conveying nothing, because unknown keys are permitted and unit is optional.
    Non-canonical values such as "µm" are the same class of failure: data that looks fine and is
    quietly wrong or quietly ignored. The validator had to add heuristic checks outside the schema
    precisely because the spec language is SHOULD (see ome/ome-ngff-validator#52).
  • google/neuroglancer#780: Neuroglancer errors parsing OME-Zarr spatial units, including "pixel",
    which OME-TIFF supports but NGFF does not enumerate — with the observation that the spec permits
    values outside the list.

Every downstream implementation (ome-ngff-validator, ome-zarr-models-py, pydantic-ome-ngff,
ngff-zarr, MoBIE/n5-ij, Neuroglancer, vizarr) currently invents its own tolerance policy. That is
the definition of a non-interoperable field.

Precedent in the spec family

RFC-4 already uses this pattern for orientation, with an explicit permissible-values table and a
controlled vocabulary. Per ome/ngff-spec#192, the RFC-4 orientation schema needs the same enum
treatment that axis unit does — which argues for fixing both with one shared artifact.

Proposed changes (for discussion)

1. Normative strength. An axis MAY contain unit; if present, its value MUST be one of the
strings enumerated for that axis type. Retain the SHOULD recommendation that physical axes carry
a unit. Optionality is preserved deliberately, since 0.6.dev4 supports unitless array coordinate
systems.

2. Expanded per-type vocabularies — canonical, singular, lowercase, UDUNITS-2-resolvable names.
Illustrative candidates only; the point of this issue is to find out which domains need what:

  • space, time — as today.
  • angledegree, radian, arcminute, arcsecond (polarization imaging, tomography tilt
    series, light-sheet/SCAPE skew).
  • spectral / channelnanometer, micrometer, angstrom (wavelength); hertz, terahertz,
    kayser (frequency/wavenumber, Raman/FTIR); electronvolt, kiloelectronvolt (energy, EM/X-ray).
  • temperaturekelvin, degree_celsius.
  • Possibly mass (dalton), concentration, magnetic field strength (tesla), pressure — driven by
    whoever shows up in this thread.
  • coordinate / displacement — clarify that these take the unit of the space they map into.

Note that angle and wavelength axis types were already raised and deferred during the 0.4 axes
discussion, so this expansion was anticipated.

3. Dimensionless / unknown. Decide explicitly between (a) omitting unit — the current de facto
meaning, and the RFC-5 authors' stated preference that "having no units key reflects that 'there are
no units' better than a placeholder" — and (b) an explicit sentinel such as dimensionless, which a
reviewer requested during RFC-5. Either way, pixel/index-style values should be handled by the
array/unitless coordinate-system mechanism already in 0.6.dev4, not by adding them to the space
enum.

4. Extension escape hatch. As with custom axis types, define how a domain uses a unit not in
the list. Candidates: a reserved namespaced prefix, a companion unitSystem/unitReference field,
or a documented "MUST be UDUNITS-2 parseable, MUST NOT be an alias of an enumerated unit" fallback
validated by pattern.

5. Machine-readable artifact. Publish the vocabularies as a single versioned schema file (e.g.
units.schema.json) that other schemas $ref, plus a plain data file (unit name, UDUNITS-2
expression, applicable axis types, description). Implementations then generate identical enums
instead of hand-maintaining lists, and ome/ngff-spec#192 is resolved uniformly across RFC-4
orientation and axis unit.

Anticipated disagreements

Raising these up front, because they are the real design tensions:

  • Closed enum vs. UDUNITS-2 grammar. An enum is trivially validatable but must be revised for
    each new community. Accepting any UDUNITS-2 expression (um, mm/s, 1e-6 m) is maximally
    expressive but requires a unit-parsing dependency in every reader and reintroduces the aliasing
    problem the enum exists to prevent. A hybrid (enum = canonical/RECOMMENDED, pattern =
    permitted-with-warning) is a plausible landing zone, but it weakens the guarantee.
  • MUST may invalidate existing writers. Some converters emit "µm" or "pixel" today. Is a
    MUST acceptable at 0.6, or should this be SHOULD plus a mandatory validator warning?
  • UDUNITS-2 vs. UCUM/QUDT/OM. UDUNITS-2 is the incumbent and CF/geoscience-aligned; UCUM is
    stronger in clinical/DICOM contexts, QUDT is more ontology-friendly for RDF-based FAIR tooling.
    Reference a second vocabulary, or provide a crosswalk table rather than switching?
  • Scope relative to RFC-4/RFC-5. Units interact with coordinateSystems, array axes, and
    RFC-4 orientation. Small self-contained "units" RFC layered on RFC-5, or folded into the axes
    work?
  • channel semantics. Attaching a wavelength unit to a channel axis may indicate the need for
    a distinct wavelength/spectral axis type.

What we are asking for now

Preliminary thoughts only — specifically:

  1. Which axis types and units does your domain need, ideally with a concrete dataset?
  2. Enum vs. grammar vs. hybrid: what can your implementation realistically enforce?
  3. MUST vs. SHOULD, and whether a shared units.schema.json would be adopted by your library.
  4. Is anyone relying on unit strings outside the current lists, beyond the "pixel" case in
    google/neuroglancer#780?

We recognize there may be substantive disagreement, which is exactly why we would like to socialize
this and gather endorsements and objections here before drafting an RFC PR. The next step is an RFC following the template, with the enumerated values, schema
changes, an alias-mapping table, and compatibility/migration behavior.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the current 0.6.dev4 axes and coordinateSystems specification, then compare RFC-4's orientation vocabulary and RFC-5's unit discussion with ome/ngff-spec#192. Gather concrete domain needs and compatibility concerns; done means an agreed units RFC defining axis vocabularies, normative strength, extension and dimensionless behavior, and the shared schema/data artifacts.

Written by the indexing model from the issue text.

Assessment

Tech stack
json
Domain
design, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.