KhronosGroup / KhronosGroup/DataFormat

RGB9E5 representation / custom float formats

Open
#13 11 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
43
Forks
8
PR merge metrics
No merged PRs in 30d

Description

I am trying to understand how to represent the RGB9E5 format as shown for:
- GL : https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_texture_shared_exponent.txt
- VK : https://www.khronos.org/registry/vulkan/specs/1.0/html/vkspec.html#textures-sexp-RGB

Note that the conversion to a value is given in both of the above as `value = mantissa x 2^(exponent - bias - bits_in_mantissa)`
For this format, bias = 15 and bits_in_mantissa = 9, and there is no implicit 1 or sign bit.

The data format spec (section 10.4, non-standard float formats) explains how to encode custom formats, and it gives an example of RGB9E5 as well (table 98), but I think I'm missing how that follows.

`sampleUpper` is supposed to be the mantissa value that represents 1 for exponent (after bias) 0.
`sampleLower` is supposed to be the mantissa value that represents 0 for exponent (after bias) 0.
For RGB9E5, substituting `0` for `exponent - bias`, we are left with `value = mantissa x 2^(0 - 9)`.
To represent the value 1, mantissa should then be `512` because `1 = 512 x 2^-9`.
To represent the value 0, mantissa should then be `0` because `0 = 0 x 2^-9`.

However, in Table 98 (the example for RGB9E5) `sampleUpper` is set to 256, not 512.
(As expected though, sampleLower = 0, bitLength = 9 and bias (sampleUpper for the exponent sample) = 15).

I wonder if Table 98 contains a mistake, or if I am missing something somewhere.

PS: if 512 is indeed the correct value, then that runs afoul of the "implicit one detection", because 512 is one larger than the representable mantissa value. Moreover, to actually represent the value 1 in RGB9E5, I believe you need to use exponent-after-bias 1 (or higher) or you run out of mantissa bits.

PS2: And by extension, there is no single representation of 1 in RGB9E5 either, because I can arbitrarily bump the exponent by one and shift the mantissa by one, and it's still a value of 1. Is there a specific "representation of 1" that should be used?

Contributor guide

No contributing guide indexed for this repository

Research direction

Read section 10.4 and Table 98 of the data format specification, then compare the RGB9E5 definitions in the linked OpenGL and Vulkan specifications. Determine whether sampleUpper and the representation-of-one guidance are consistent, and document or correct the specification accordingly.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.