cdisc-org / cdisc-org/DDF-RA

Issues with use and validation of API-defined data structures

Open
#689 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
52
Forks
5
PR merge metrics
No merged PRs in 30d

Description

While the USDM IG seems to refer to the API as only having a single structure/schema, the API specification actually contains a number of endpoints for loading and retrieving USDM data, and the structure of the data that is either required or returned is not the same for all endpoints - as shown by this extract:
```json
{
"POST /v4/studyDefinitions": {
"summary": "Create a study",
"description": "Create an entire study including all child element with a single post",
"schema": {
"$ref": "#/components/schemas/Wrapper-Input"
}
},
"PUT /v4/studyDefinitions/{studyId}": {
"summary": "Update a study",
"description": "Update an entire study including all child element with a single put",
"schema": {
"$ref": "#/components/schemas/Wrapper-Input"
}
},
"GET /v4/studyDefinitions/{studyId}": {
"summary": "Return a study",
"description": "Return an entire study including all child elements",
"schema": {
"$ref": "#/components/schemas/Wrapper-Output"
}
},
"GET /v4/studyDefinitions/{studyId}/history": {
"summary": "Returns the study history",
"description": "Returns the history for the specified study",
"schema": {
"type": "array",
"items": {
"$ref": "#/components/schemas/Wrapper-Output"
}
}
},
"GET /v4/studyDesigns": {
"summary": "Study designs for a study",
"description": "Returns all the study designs for a specified study.",
"schema": {
"type": "array",
"items": {
"anyOf": [
{
"$ref": "#/components/schemas/InterventionalStudyDesign-Output"
},
{
"$ref": "#/components/schemas/ObservationalStudyDesign-Output"
}
]
}
}
}
}
```
- The endpoints for the creation, update and retrieval of a single study definition all expect/provide the same data structure, which is based on the `Wrapper-Input` / `Wrapper-Output` schemas. These two schemas are identical in terms of data structure, with the `study` attribute of the `Wrapper` class containing a study definition that aligns closely with the `Study` class and its descendants, as defined in the USDM model.
- The "history" endpoint returns a list/array of study definitions, each of which has a structure based on the `Wrapper-Output` schema.
- The "studyDesigns" endpoint returns a list of study designs, each of which has a data structure based on either the `InterventionalStudyDesign-Output` schema or the `ObservationalStudyDesign-Output`.

The USDM IG references to the API seem to be based only on the data structure defined by the "Wrapper" schemas used for the creation, update and retrieval of a study definition, and there is no mention of the list/array structure for the "history" and "studyDesigns" endpoints. Also, while the USDM IG [recommends](https://wiki.cdisc.org/display/USDMIGv4/Additional+Attributes+and+Required+Content) that "There is only 1 StudyVersion", the model- and API-defined cardinality of the `Study.versions` attribute is 0..*.

It is not clear which of the data structures defined in the API should be supported for CORE conformance rules, and whether/how multiple study versions (multiple instances of the `StudyVersion` class) should be handled for CORE validation.

Currently, the CORE engine will only recognize data as being USDM data if its structure is based on the "Wrapper" schema (it must have a `study` attribute at the top level). If the CORE engine recognizes a JSON file as containing USDM data, it will process the whole file - even if it contains multiple instances of the `StudyVersion` class. However, most - if not all - CORE rules have been defined/implemented based on the assumption that there will only be a single study version in the data being validated. If multiple study versions are validated, it's likely that validation will produce unexpected results.

#### Questions about CORE implementation:
- Should CORE support be extended to cover other data structures defined in the API schema? It's unlikely that there'd be much need for validation of a study's history, but it's conceivable that you might want to validate extracted study designs. If not, I think we should document the restrictions - but where?
- How should we deal with the potential for multiple study versions?
- Should the API cardinality for `Study.versions` be restricted to allow only one study version? It might seem strange to restrict an array attribute to `maxItems = 1` (if this is even possible), but otherwise there'd be a bigger difference between the API and model.
- Should the CORE engine deal with this:
- By raising an error and failing if more that one version is found?
- By only processing one version - if so, how would this be chosen?
- Last in `versions` list?
- Highest `StudyVersion.versionIdentifier` value? Except this is a string.
- By running all rules separately for each version - which seems pointless (why would you want validation of previous versions?)
- Should individual rules to defined/implement to handle the possibility of multiple versions? This might involve a lot of rework, but could possibly be accommodated more easily in the move to JSONata.

#### Questions about API structure/usage:
- What is the expectation for use of the "history" endpoint?
- It seems strange to repeat the `Wrapper` attributes (`usdmVersion`, `systemName` and `systemVersion`) with each of the study definitions in the list. If the structure of the study definition is defined by the API schema, then all study definitions must have the same USDM version and I'd guess that the entire history would be produced at one time by the same version of the same system.
- How does the list of study definitions align with the use of study versions? Should each study repeat contain only a single study version, or should there be progressively more study versions for each of the study repeats as they build up over time?
- The `InterventionalStudyDesign-Output` and `ObservationalStudyDesign-Output` schemas are the same ones that are used within the `Wrapper` schemas - they only contain a subset of the available information for a study and may include references to object definitions that are not stored within the study design (e.g., `studyInterventionIds`, `biomedicalConceptIds`, `eligibilityCriterionItemIds`). These references will be invalid/meaningless without access to the referenced definitions. Is this OK, or should specific schemas be defined where external references are replaced with the corresponding definitions?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.