KhronosGroup / KhronosGroup/OpenXR-SDK-Source

hello_xr colors/non-linearity seems to be ill-defined and inconsistent, especially across different swapchain formats and headsets

Open
#467 7 comments 2 reactions 0 assignees View on GitHub
synced to gitlab
Dominant language
Python
Stars
825
Forks
306
Avg merge
1d 23h
Merged PRs (30d)
1

Description

### Big-picture issue:
I recently ran into a curious discrepancy between all of the headsets I have, as part of my attempts to add HDR support to the ALVR project, verifying my changes has no regressions:

![headset_swapchains](https://github.com/KhronosGroup/OpenXR-SDK-Source/assets/1224096/a379cfe4-265a-4785-bc3d-9df2a10c3d1d)

These screenshots are of `hello_xr` depicted as they appeared to my own eyes in-headset, the key feature being the background darkness. To obtain them, I checked out the current main branch as of this issue, and forced the list of app-supported formats to just the described formats, only `GL_RGBA16F` and then only `GL_SRGB8_ALPHA8`. The headsets all say they support *both* of these swapchain formats. These headsets are all OpenXR certified, that I'm aware at least.

[XrSwapchain](https://registry.khronos.org/OpenXR/specs/1.0/man/html/XrSwapchain.html) seems to be too ambiguous, and the hello_xr conformance testing has not picked up on these different interpretations of XrSwapchain. That I can tell, there are 3 different assumptions I can see being made here:
- **Meta**: "Images submitted in sRGB color space must be created using an API-specific sRGB format" refers to the more-tightly defined specification of ITU-R BT.709, *not* non-linear RGB color. `DXGI_FORMAT_R8G8B8A8_UNORM` is not able to represent the full sRGB color space well. The compositor should only read linear colors, and gamma correction is done exclusively by the compositor. The underlying data is irrelevant, sRGB textures are just a compression format.
- **Valve**: "Images submitted in sRGB color space must be created using an API-specific sRGB format" means that if an application is submitting gamma-corrected color, it is signaled by creating a swapchain with an sRGB format. The application may be writing linear colors to an sRGB render target, or gamma-corrected colors to an RGB render target. The underlying data is relevant, and sRGB texture formats change how the frame is displayed.
- **Pico**: "Images submitted in sRGB color space must be created using an API-specific sRGB format" means that linear RGB colors must be corrected to be non-linear RGB (??)

## hello_xr colors as linear RGB:

in `src/common/xr_linear.h`, there are the following definitions for grey:
```
static const XrColor4f XrColorLightGrey = {0.7f, 0.7f, 0.7f, 1.0f};
static const XrColor4f XrColorDarkGrey = {0.3f, 0.3f, 0.3f, 1.0f};
```
in web non-linear sRGB color, these would be #959595 and #dadada. Which, to me personally, is more of a light gray and a white in-headset. But, then again, Magenta is also labeled as `XrColorPurple`. The OpenXR spec has [a very specific definition](https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#colors) for XrColor4f, which is that it only holds linear RGB unless otherwise specified.

- ![#959595](https://placehold.co/15x15/959595/959595.png) `#959595`
- ![#dadada](https://placehold.co/15x15/dadada/dadada.png) `#dadada`

There's also this `SlateGrey` color in `hello_xr/options.h`:
```
static const std::array SlateGrey{0.184313729f, 0.309803933f, 0.309803933f, 1.0f};
```
in web non-linear sRGB color, it's #779797. This is also a very light grey-blue, instead of a dark grey-blue.

- ![#779797](https://placehold.co/15x15/779797/779797.png) `#779797`

To provide an extra opinion on what dark grey, light grey and slate gray look like, here are GitHub's LaTeX interpretations:

$${\color{DarkGrey}Dark Grey █ }$$

$${\color{LightGrey}Light Grey █ }$$

$${\color{LightGrey}Slate Grey █ }$$

## hello_xr colors as non-linear RGB:

Interpreting all of those same colors as *gamma-corrected* colors seems to yield colors closer to what they are labeled. Dark grey would be #4d4d4d, light gray #b2b2b2, slate gray would be #2f4f4f.
- ![#4d4d4d](https://placehold.co/15x15/4d4d4d/4d4d4d.png) `#4d4d4d`
- ![#b2b2b2](https://placehold.co/15x15/b2b2b2/b2b2b2.png) `#b2b2b2`
- ![#2f4f4f](https://placehold.co/15x15/2f4f4f/2f4f4f.png) `#2f4f4f`

Additionally, [at some point in the past](https://github.com/KhronosGroup/OpenXR-SDK-Source/commit/6d572b75a5e0c49fa56207839bdac662facb9ecc#diff-17d510a2258925f58acaf336e7ac9c72a725838956b8ca4751a114fc985765a5L16), the same slate gray was once described as `DarkSlateGray`. This name is still used by the current Pico 4 SDK sample, interestingly.

## Takeaways:
- CTS should include testing *all* runtime-supported texture formats
- If Valve's OpenXR implementation is correct, `hello_xr` should gamma correct non-linear texture formats.
- If Pico's OpenXR implementation is correct, `hello_xr` should gamma correct linear texture formats.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the XrSwapchain specification and the color definitions in src/common/xr_linear.h and hello_xr/options.h. Reproduce hello_xr with GL_RGBA16F and GL_SRGB8_ALPHA8 across the reported headsets, then compare the runtime behavior with the CTS coverage. Done means the intended color-space semantics are clarified and the affected sample or conformance tests have an agreed change.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
ar-vr-xr, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.