Handling of HDR color spaces
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Feature Idea
I have created a model which can produce images in Rec. 2100 PQ to natively create HDR images. However for this to work well, two things are missing:
1. A way for the model to indicate the images it is outputting is not sRGB
2. A way to include that color space information in the saved files, preferably automatically
I think it would make a lot of sense to include some meta-data in the models. My primary motivation for creating this issue is to know how I could tag my models such that it could be potentially supported in the future, even if there is no interest to support HDR for now.
I see that there are several difficulties with supporting HDR:
- PNG has support for HDR through a new iCIP chunk, but pillow requires a hack to actually write it: https://gist.github.com/zhuowei/96b6e184bcf2de64433fbb86e15f7762
- JPG can be supported through Googles UltraHDR extension, but I'm currently not aware of a Python library for writing these
- AVIF/HEIC/JPEG XL supports two different way of storing HDR (10-bit vs. gain maps)
- Currently only Chromium based browsers support HDR AFAIK.
And then there is the question on what part of the model should decide what the color space is. Both the UNET, Lora, and VAE can affect the final image. My approach ended up by fine-tuning a VAE to remap the SDR latents to Rec. 2100 PQ and then using a Lora to push it towards creating images outside the normal SDR range. (Only acting on the UNET made the results very finicky whenever the prompt affected colors and/or brightness.)
But with all that said, I personally think HDR support would be very exciting. Here are some examples:
https://civitai.com/images/52054640 (PNG 8-bit)
https://civitai.com/images/52208460 (PNG 8-bit)
https://civitai.com/images/51872956 (UltraHDR tweaked in Adobe Camera Raw)
### Existing Solutions
_No response_
### Other
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.