AcademySoftwareFoundation / AcademySoftwareFoundation/openexr

[Docs] hex values for pixel data in the "Sample File" in the docs don't match the floating point values

Open
#1,433 5 comments 0 reactions 0 assignees View on GitHub
needs attention
Dominant language
C
Stars
1.8k
Forks
700
Avg merge
3d 12h
Merged PRs (30d)
29

Description

On [this page ](https://openexr.com/en/latest/OpenEXRFileLayout.html#sample-file) the hex data listed for the file doesn't seem to match the floating point values written below it. Neither the FP16 or the FP32 values seem to match exactly their hex representations : eg.

The first FP32 value (pixel (0,0) in the Z channel) is listed as

5C28813A
0.000985395

but the little endian IEEE-754 representation of 0.000985395 should be 0x **5F** 28 81 3A unless I'm very mistaken.

The first non zero FP16 value (pixel (1,0) in the G channel) is listed as :

5429
0.042

but by my calculations 0.042 in FP16 should be 0x **60** 29

There seem to be two possibilities for how this has happened : either this is an exotic binary FP representation (which seems unlikely?), or the floating point numbers in the table represent some original pixel data that has undergone some kind of transformation prior to being encoded (maybe log-> linear ?)- in which case the "sample file" documentation is a bit misleading / confusing because all the other values in the description relate to the data in the file, rather than original pixel data.

If this discrepancy in the floating point encoding represents something deliberate it should probably be noted somewhere in the document - if it is noted somewhere, then I can't find it. I've been using this "sample file" data in test to check conformance of a simple Scala EXR codec, and it's taken me many hours to work out why its been failing - hopefully someone else can be saved the frustration if the docs are updated.

Contributor guide

Open the contributing guide

Research direction

Start with the Sample File section of docs/OpenEXRFileLayout.html and compare the listed FP16 and FP32 values with their hex representations. Determine whether the discrepancy is intentional, then correct the sample values or document the transformation clearly so the file data and displayed numbers are unambiguous.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.