CesiumGS / CesiumGS/wetzel

Consider a refactoring for the tests

Open
#80 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
137
Forks
52
PR merge metrics
No merged PRs in 30d

Description

The approach for the tests in wetzel is explained in a [comment in `test.js`](https://github.com/CesiumGS/wetzel/blob/82890df68ae4c24c5679196c06e3afccd6673a30/test/test.js#L13) : Based on a set of example schemas, the tests consist of automatically generating the property reference in different configurations, and comparing the resulting files with the "golden" output that is checked in into the repository. The exact inputs and parametrizations are summarized in the [`index.json`](https://github.com/CesiumGS/wetzel/blob/82890df68ae4c24c5679196c06e3afccd6673a30/test/test-schemas/index.json).

The example schemas for the tests are apparently based on glTF. But additional features have been added to some of these schema files, and it is not always clear which aspect of the schema files are supposed to cover which functionality. One specific example: The `image.schema.json` is _largely_ a glTF image, but contains some [test for fractions](https://github.com/CesiumGS/wetzel/blob/82890df68ae4c24c5679196c06e3afccd6673a30/test/test-schemas/v2020-12/image.schema.json#L38). I think that it could make sense to break these tests down into smaller pieces that have a "semantic meaning". For example, these 'fractions' could be tested with a dedicated `fractions.schema.json`.

Other aspects that could be covered with dedicated tests could be

- circular references
- nested type definitions
- the handling of `additionalProperties`
- details about strings (patterns, lengths, formats)
- Maybe important: Subtle differences regarding the JSON schema version. For example, the change of the meaning of `minimum`/`exclusiveMinimum` that was done after Draft 04 - see [Range](https://json-schema.org/understanding-json-schema/reference/numeric.html#range) for details

The advantage would be that these schemas can be documented via the `description`, clearly explaining which aspect of the schema is tested there, and that it could be easier to apply specific changes or add and test specific new functionality.

There are some questions that will certainly come up either in this process, or in the medium term in general:

- Which parts of JSON Schema are supposed to be supported in the first place?
- What should the generated documentation look like, exactly?
- In how far should details of the generated result be configurable (e.g. via CLI parameters)?

But I think that a few, first steps for creating such a set of example schemas could be done independently.

(Note: all this could be done as a pure _addition_ to the current tests. But we might as well try to "clean up" the current schemas so that they more closely resemble the relevant parts of the _current_ glTF schema, and use this part as a more coarse-grained "integration test". And these changes would _solely_ be on the test schemas, and _not_ affect any part of the actual schema generation code, just to avoid regressions)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the testing approach comment in test/test.js and the parametrizations in test/test-schemas/index.json. Review the existing schemas, including test/test-schemas/v2020-12/image.schema.json, and identify which behaviors should become focused examples. Done would mean adding semantically documented schemas and tests for agreed cases without changing schema generation code.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.