KhronosGroup / KhronosGroup/glTF
Better define supported JPEG images
- Dominant language
- HTML
- Stars
- 7.8k
- Forks
- 1.2k
- Avg merge
- 17h 26m
- Merged PRs (30d)
- 5
Description
glTF 2.0 specification requires that implementations support JPEG images compatible with the JFIF format.
The normative definition of JPEG compression is [ITU-T Rec. T.81](https://www.itu.int/ITU-T/recommendations/rec.aspx?id=2633) (or identical [ISO/IEC 10918-1](https://www.iso.org/standard/18902.html)). This standard defines various lossy and lossless compression techniques for abstract "image components", i.e., it does not say how these components should be mapped (or converted) to regular RGBA color channels that glTF expects. Also it does not require all implementations to support all encoding methods.
The JFIF standard is defined in [ITU-T Rec T.871](https://www.itu.int/ITU-T/recommendations/rec.aspx?id=11319) (or identical [ISO/IEC 10918-5](https://www.iso.org/standard/54989.html)). It imposes the following restrictions on JPEG data:
- There must be an APP0 marker saying `JFIF` right after the SOI marker.
- Image components must be 8-bit.
- There must be either 1 or 3 image components.
- Images with 1 component are grayscale, i.e., Y=R=G=B.
- Images with 3 components encode YCbCr values. YCbCr to RGB conversion is defined.
The JFIF standard also suggests use of JPEG baseline process for maximum compatibility.
---
The suggestion to use _only_ the baseline process is not very practical. For instance, the Sample-Assets repo has 585 JPEG images and 125 out of them use progressive encoding instead of baseline without any issues. That said, there are certain JPEG encoding modes with very little application support. As of today, the glTF 2.0 specification does not explicitly disallow them thus an asset with such images is formally correct but unusable in practice.
Problematic features:
1. Arithmetic coding. Not supported in Chrome and Firefox; some variants work in Safari.
2. Lossless encoding. Not supported in Chrome and Safari; some configurations may work in Firefox.
3. Hierarchical mode and DNL markers. Not supported in any browser, rarely supported by libraries.
---
**Proposed glTF 2.0 update** (for the section 2.6)
> - JPEG images **MUST** be compatible with JPEG File Interchange Format, namely:
> - JPEG images **MUST** have a `JFIF` APP0 marker right after the SOI marker.
> - JPEG images **MUST** have 1 or 3 image components with 8 bits per component
> - Additionally, for application compatibility:
> - JPEG images **MUST** use Huffman coding.
> - JPEG images **MUST** use Baseline DCT, Extended sequential DCT, or Progressive DCT signalled by SOF0, SOF1, or SOF2 markers respectively.
> - JPEG images **MUST NOT** use the hierarchical mode of operation.
> - JPEG images **MUST NOT** contain a DNL marker.
/cc @javagl @emackey
Contributor guide
Assessment
This issue has not been assessed yet.