imazen / imazen/zenpixels

Track: native f16 pixel support (waiting on Rust stable `f16`)

Open
#23 2 comments 0 reactions 1 assignee Claimed by @lilith View on GitHub
enhancement
Dominant language
Rust
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Status

Tracking issue. **Do nothing until `f16` stabilizes in Rust stable.** As of April 2026, `f16` is nightly-only behind `#![feature(f16)]` — see [rust-lang/rust#116909](https://github.com/rust-lang/rust/issues/116909). When it lands, execute the plan below.

Rationale for waiting: unconditional dependency on the `half` crate is workable but adds a compile-time dep, a parallel type universe (`half::f16` vs `f16`), and trait-impl churn on the boundary. Native `f16` removes all three. The cost of waiting is zero — nothing currently produces or consumes f16 pixels in the zen ecosystem (see "Current blocked consumers" below), so there's no pressure to ship behind `half`.

## Current state in zenpixels

- `ChannelType::F16 = 5` already exists in `zenpixels/src/descriptor.rs:27` on the `#[non_exhaustive]` enum, with `is_f16()` and `byte_size()=2` helpers. It's constructable but no conversion path exists.
- Cost model in `zenpixels-convert/src/negotiate.rs:605,696–731` already scores F16 conversions (F16↔F32 = 15, F16→U8 = 30, depth bits = 11, F32→F16 precision loss = 20).
- `half = \"2\"` is in `zenpixels-convert/Cargo.toml` dev-deps for perceptual-loss tests only.
- CMS layer in `zenpixels-convert/src/cms_moxcms.rs:251` already routes F16 through the f32 transform.

So: declared everywhere, implemented nowhere.

## Plan outline (execute when `f16` stabilizes)

### Layer 1 — zenpixels public API (additive, non-breaking)

`PixelFormat` (enum at `descriptor.rs:1300–1320`) is `#[non_exhaustive]`, so new variants are safe per our 0.2.x versioning policy. Add:

- `RgbF16`, `RgbaF16` — TIFF `SampleFormat=IEEEFP BitsPerSample=16`, Ultra HDR gain-map color, JPEG XL modular f16, future OpenEXR
- `GrayF16`, `GrayAF16` — single-channel Ultra HDR gain maps, HDR luma/depth
- Matching `pub const` presets (`RGBF16_LINEAR`, `RGBF16_SRGB`, `GRAYF16_LINEAR`, …) beside existing F32 ones at `descriptor.rs:741–947`

**Explicitly NOT adding:** `OklabF16`, `OklabaF16`. Oklab is compute-only in zen — no codec produces or consumes it on the wire, and f16 Oklab in a multi-stage filter chain accumulates error (see precision analysis below). YAGNI gate applies.

**Also probably not adding:** `GrayAlphaF16` as a concrete `Pixel`-impl'ing type in `pixel_types.rs`. Only promote if a real consumer surfaces; descriptor-only bytes are enough for most paths.

`with_*` relabel APIs are depth-agnostic — no changes needed.

Open question: does `PixelBuffer` / `Pixel` in `buffer.rs` generalize over a lane type today, or is F16 a new row-format path? This determines whether Layer 1 is a half-day or two days.

### Layer 2 — zenpixels-convert (the real work)

Planner at `convert.rs:973 depth_steps()` currently matches `{U8, U16, F32}` only; F16 falls through to `NoPath`. Needed:

1. **6 new `ConvertStep` variants** in `convert_kernels.rs:38–157`: `F16ToF32`, `F32ToF16`, `U8ToF16`, `F16ToU8`, `U16ToF16`, `F16ToU16`.
2. **~12 depth-conversion kernels** mirroring the existing int↔f32 set:
- Basic: `f16_to_f32`, `f32_to_f16`, plus via-f32 combinations for u8/u16
- Fused TF variants: `srgb_f16_to_linear_f32`, `linear_f32_to_srgb_f16`, `pq_f16_to_linear_f32`, `linear_f32_to_pq_f16`, `hlg_*` equivalents
3. **F16 arms** in `depth_steps()`, `int_to_f32_step()`, `f32_to_int_step()` — currently panic on F16.
4. **`MatteComposite`** (`convert_kernels.rs:430–480`) — no F16 arm today; falls through to empty `_ => {}`. Matte math already operates in f32, so it's straightforward.
5. **Fix depth-reduction policy bug at `converter.rs:178–180`**: gate is `byte_size()` differing, so F16↔U16 would silently skip policy checks (both 2 bytes). Needs an explicit \"precision changed?\" predicate. This is a bug today — F16 just exposes it.
6. **Dependency decision: add `f16` directly, drop `half` from dev-deps** once stable `f16` lands. If the stabilization happens before we're ready to execute, we can bridge via `half` — but prefer native.

TF code (sRGB/PQ/HLG curves, gamut matrices, Oklab math) needs **no** changes: the pipeline pivots through f32 linear at `convert.rs:278–281`, confirmed. Only entry/exit depth conversions need f16 knowledge.

### Layer 3 — SIMD (optional, measure first)

Scalar conversion is correct; hardware acceleration is the polish:
- **F16C** (Haswell+): `vcvtph2ps` / `vcvtps2ph`, 8 lanes
- **AVX-512 FP16**: 32 lanes
- **ARMv8.2 FP16**: `vcvt_f32_f16`, `vcvt_f16_f32`

Wire through archmage alongside existing `garb` / `linear-srgb` kernels. Ship scalar first; add hardware dispatch when profiling shows it matters.

**No `f16` feature flag.** Middleman tax (zencodec, zenpipe, imageflow all needing forwarding) outweighs the compile-time savings, and CLAUDE.md's \"feature flags are part of the API\" rule applies. If hardware-specific SIMD compile time becomes a problem, an opt-out `f16-simd` feature is the narrower place to put it — not on the type system.

## Precision analysis (why the scope decisions above)

**f16 precision, as integer equivalent over [0, 1]**:

| Value range | f16 ULP | Equivalent |
|---|---|---|
| [0.5, 1.0] | 2^-11 ≈ 4.9e-4 | u11 |
| [0.25, 0.5] | 2^-12 | u12 |
| [0.125, 0.25] | 2^-13 | u13 |
| [1.0, 2.0] HDR | 2^-10 ≈ 9.8e-4 | u10 |
| [2.0, 4.0] HDR | 2^-9 ≈ 2e-3 | u9 |
| [4.0, 8.0] HDR | 2^-8 ≈ 4e-3 | u8 |

f16 keeps 11 bits of *relative* precision; uN keeps N bits of *absolute* precision. They intersect near 1.0. For HDR (PQ/HLG output) precision drops fast — at peak luminance you're at u8–u9, which is why HDR pipelines need careful audit before allowing f16 intermediates.

**Error accumulation over 4–5 chained ops**:

| Op class | Error per op | 4–5 ops total |
|---|---|---|
| Well-conditioned FMA (exposure, saturation) | ~0.5 ULP | ~1 bit lost |
| Mild cancellation (contrast) | ~0.5–1 ULP | ~1–2 bits lost |
| **Catastrophic cancellation** (clarity, unsharp, detail extraction) | 2–8 ULP | **3–5 bits lost** |

Implication: f16 is safe for simple 4–5 op SDR chains; unsafe the moment detail-extraction or HDR tone-mapping enters the pipeline. \"Don't compute in f16 Oklab\" is not a convention — it's a numerical requirement for any pipeline that includes an unsharp/clarity step.

## Current blocked consumers (dependents that would benefit)

Each already touches f16 via the `half` crate today and collapses to f32 at the zenpixels boundary because there's nowhere to put it:

| Crate | Wire format | Current workaround | Benefit |
|---|---|---|---|
| **zentiff** | TIFF `SampleFormat=IEEEFP BitsPerSample=16` | `decode.rs` forces `h.to_f32()` | High — near one-file change to surface native bit depth |
| **zenjpeg** (Ultra HDR) | f16 gain map in APP2 (ISO 21496-1) | `ultrahdr/encode.rs` reads via `half::f16::from_bits().to_f32()` | Gain-map tone-mapping in f16 halves memory |
| **ultrahdr** | ISO 21496-1 gain maps | Decodes f16 → f32 immediately | Same as zenjpeg |
| **image-tiff** (upstream) | TIFF FP16 | Already uses `half`; zentiff is the surfacing layer | Enabler for zentiff |
| **heic**, **zenavif** | HEIF/AVIF 10/12-bit + HDR gain maps | Decode to u16 YUV, tone-map in f32 | Medium — f16 as HDR tone-map intermediate |
| **zenjxl** | JPEG XL modular mode | Not yet optimized | Modular f16 passthrough |

No benefit from f16: zenpng, zenwebp, zengif, zenbitmaps (PFM is f32 by spec), zenresize, zenfilters, zenblend, zenquant, zensim, fast-ssim2, butteraugli. OpenEXR would be the single biggest consumer if we ever add it.

## Non-breaking status

All proposed changes land inside our 0.2.x tolerated-break window or are purely additive:
- New `PixelFormat` variants on `#[non_exhaustive]` — non-breaking
- New `pub const` presets — non-breaking
- New `ConvertStep` variants (also `#[non_exhaustive]`) — non-breaking
- New kernel functions — non-breaking
- `converter.rs:178–180` bug fix — behavior change that catches precision loss previously silently allowed; document as a tolerated fix

No 0.3.0 bump required.

## Estimated effort once stable `f16` lands

- Layer 1 (zenpixels types + presets): 0.5–2 days depending on `PixelBuffer` generalization question
- Layer 2 (zenpixels-convert kernels + planner + policy bug): ~1 day scalar-only
- Round-trip tests modelled on `roundtrip.rs:238/279/319/359`, tolerance ~2 f16 ULP: ~2 hours
- zentiff surfacing: separate issue in that repo
- ultrahdr/zenjpeg gain-map buffers: separate issues
- SIMD dispatch: deferred, measure first

**Total for zenpixels + zenpixels-convert: ~2 days of focused work** once the Rust primitive is stable and we have downstream pressure from at least one dependent.

## Checklist to revisit this issue

- [ ] `f16` stable in a released Rust (check [rust-lang/rust#116909](https://github.com/rust-lang/rust/issues/116909))
- [ ] At least one dependent (most likely zentiff) has concrete pressure to surface f16
- [ ] MSRV bump acceptable or staged feature is available

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.