CesiumGS / CesiumGS/3d-tiles-validator

Try to continue structural validation in case of non-critical errors

Open
#360 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
474
Forks
156
PR merge metrics
No merged PRs in 30d

Description

There is one overarching question for the general approach of the validation process:

**When should the validator stop validating?**

The [earliest incarnations of the 'validator'](https://github.com/CesiumGS/3d-tiles-validator/blob/7738248755bcc6dc52eb7fc78fc954b14157957a/validator/lib/validateTileset.js#L47) did not bother with this: As soon as one (even trivial) "error" was found, the process bailed out immediately, throwing an `Error`. So when a tileset had a missing `geometricError` and a missing `asset`, then the first one was detected by the validator, then had to be fixed, and the missing asset was detected only at the _second_ validation run.

The new/current state of the validator tries to be eager and resilient: It tries to gather as many errors as possible, and continue validation as long as possible, in order to compile a validation report that is 'as complete as possible'.

But still, there _are_ places where the validation has to stop because of an error. There are obvious reasons for that. When the root of an implicit tileset defines `availableLevels: -999`, then the implicit tile hierarchy cannot be validated. Other cases may be a judgement call. When a tileset uses metadata, but does not define a `schema`, then the validator cannot sensibly validate the metadata that is contained in the tiles. It could now try to validate the tile hierarchy itself, and skip the metadata validation part. But for some ~"things" (invalid elements), trying to keep track of _what is valid or not_ and _what can be validated or not_ (as a consequence of that) could be prohibitively complex.

One very specific case: When the validator determines the root of an implicit tileset to be invalid, then it does not validate the implicit tile hierarchy. But there are forms of errors that would not necessarily have to prevent a further validation. For example, when the implicit root contains _metadata_. This is invalid, but the remaining structure _could_ still be validated. Maybe this and similar cases can be identified, and warrant a special handling, treating these errors as "non-critical" for the validation process itself.

(One could put an arbitrary amount of effort into this. And one could argue about the _benefit_ that this brings in practice. But it's always a trade-off, and there might be some low-hanging fruits in this regard)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked historical validator/lib/validateTileset.js entry point, then trace the current validation flow for implicit tilesets and root errors. Identify which invalid conditions prevent independent checks from running. Done should include an agreed policy for non-critical errors and tests showing that safe validation continues while dependent validation still stops.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.