KhronosGroup / KhronosGroup/glTF

Clarification of "two implementations" requirement for ratification

Open
#2,542 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
7.8k
Forks
1.2k
Avg merge
17h 26m
Merged PRs (30d)
5

Description

### Summary (and Disclaimer)

There is a requirement for ratification that is sometimes informally summarized as ~_"there must be two implementations (of that 'thing' that should be ratified)"_. In the context of glTF, this is mainly relevant for _extensions_. But it is not clear what this should mean.

This is, in many ways, not really important for me. I'm certainly not a fan of ~"over-formalizing processes", to the point where "following the process" implies more work than _just doing it_. But these questions are not so much about the process (_how_ things are done), but rather about the tasks (_what_ has to be done). Given the implications for a formal ratification, there should be clear answers to that.

### Context

I brought up the question of what "an implementation" should mean, recently [in comment in the `KHR_gaussian_splatting PR`](https://github.com/KhronosGroup/glTF/pull/2490#issuecomment-3589363862), and a [comment in the `KHR_meshopt_compression` PR](https://github.com/KhronosGroup/glTF/pull/2517#pullrequestreview-3548434351). It is somehow related to (not so recent) attempts to [clarify some aspects of the extension development process](https://github.com/KhronosGroup/glTF/pull/2225), even though that PR is closed in the meantime, and I omitted formal details about the ratification process there.

I think it could make sense to funnel these considerations and questions into a dedicated issue.

### The Core

The ratification requirement that "there must be two (independent) implementations" probably has its roots in the [Khronos Group Operational Guidelines (PDF)](https://www.khronos.org/files/working-group-guidelines.pdf), which lists...

> 3. Specification Development
> 3.1. Ratification Criteria and Process
> [...]
> Ideally, when a specification is submitted for ratification, the Working Group will be aware of at least two independent conformant implementations of the specification
> [...]
> Pragmatically, final conformant implementations and conformance tests are interdependent and insisting on conformant implementations and proven tests before ratification could delay the specification to the extent of damaging market acceptance. In which case, the Board typically look for the following milestones to be met before ratification:
> - The Working Group is confident that all areas of the specification have been independently implemented twice so that ‘specification bugs and flaws that become obvious through implementation’ have been discovered. The implementations do not have to be complete in themselves - multiple partial implementations from different members can be used to generate specification coverage, additionally, the implementations do not have to be released, or announced or conformant – or even available to the Working Group – working on trust to take implementers at their word.

The vagueness and non-binding nature of these "requirements" justify the informal nature of the summary. With words like "ideally", "confident", "partial", and "trust" (!), there are no explicit, actual "requirements" listed for these "implementations".

Some of that vagueness can be justified, given how broadly this should be applicable. For something like an OpenGL extension, saying "It works on AMD and NVIDIA cards" may be enough. But for things like glTF extensions, there are more cases to consider.

### Cases

A glTF extension can just add a single value like `emissiveStrength: number` to some PBR material property. Parsing the JSON is beyond trivial. For renderers, this might boil down to inserting some `... * emissiveStrength` in some shader code. (Yes, it may be more complicated, but conversely, it could often really just mean that an _existing_ configuration option of a renderer is _filled_ with the value from that glTF extension).

A glTF extension that defines a specific binary bitstream is an entirely different category. Examples here are the extensions for Draco- or meshopt compression. The _specification_ of the bitstream itself is already a challenge. It is virtually impossible to look at a specification and say with absolute certainty that it is possible to implement a decoder for this bitstream, _from scratch_, just based on this description. And implementing a _decoder_ is usually still _far_ easier than implementing an _encoder_.

There are many in-between cases. For example, I brought up the question about what "an implementation" is in the `KHR_gaussian_splatting` PR. Should it be a _reader_ or a _writer_ or a _renderer_? Every existing glTF loader already _is_ an implementation for that, assuming that every loader just loads the accessors with their data in a generic form. It doesn't really have to _do_ anything with this data. Similarly, most _writers_ can probably already _write_ generic accessors, and the question is rather where the splat data is coming from, and who is responsible for bringing it into the exact shape that is required for that extension. (But once it _is_ in that shape, it can trivially and generically be written). The JSON part is negligible here. But there are many _really_ tricky questions about how _a renderer should render these splats_ in order to be a "conformant" implementation. (These questions are probably too specific to be considered here).

### Details

The final (nearly philosophical) consideration could be on the level of the language. The linked guidelines talk about "conformant" implementations (but also "partial" implementations), and about _"independent"_ implementations. And I think that the term _"independent"_ could be important.

Emphasizing the point from the guidelines document:

> The Working Group is confident that all areas of the specification have been **independently** implemented twice so that **‘specification bugs and flaws that become obvious through implementation’** have been discovered.

The goal of finding specification bugs (mainly inconsistencies and ambiguities) can only be accomplished when the implementations are really _independent_ - roughly meaning that they have been implemented ~"by two different people, _only_ based on the specification". Just having implementations (by the same person) in two different programming languages is certainly not enough. And _porting_ an existing implementation from one language to another also does not help to identify bugs and shortcomings of the specification document itself - it may not even have been consulted during that porting process. (The case where the code was ported with any form of automated tool obviously falls into the same category).

Contributor guide

Open the contributing guide

Research direction

Start with the ratification criteria in the Khronos Group Operational Guidelines and the linked glTF discussions in PRs 2490, 2517, and 2225. Determine how “implementation,” “independent,” “partial,” and “conformant” should apply to glTF extensions. Done means the project has an agreed clarification of the two-implementation requirement.

Written by the indexing model from the issue text.

Assessment

Domain
documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.