dandi / dandi/dandi-schema

Validation has weakened in context of vendorization

Open
#368 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7
Forks
15
PR merge metrics
No merged PRs in 30d

Description

For example, if you compare the DOI validation in schema version [0.6.0](https://github.com/dandi/schema/blob/9249a0cbf9ae413d2219291980ee9cb2e7e6246c/releases/0.6.0/published-dandiset.json#L258):

```json
"doi": {
"title": "DOI",
"pattern": "^10.(48324|80507)/dandi\\.\\d{6}/\\d+\\.\\d+\\.\\d+",
"readOnly": true,
"nskey": "dandi",
"type": "string"
}
```

with the same in version [0.7.0](https://github.com/dandi/schema/blob/9249a0cbf9ae413d2219291980ee9cb2e7e6246c/releases/0.7.0/published-dandiset.json#L1562):

```json
"doi": {
"default": "",
"nskey": "dandi",
"pattern": "^(10\\.\\d{4,}/[a-z][-a-z]*\\.\\d{6}/\\d+\\.\\d+\\.\\d+|)$",
"readOnly": true,
"title": "DOI",
"type": "string"
}
```

you can see that the validation moved from "has a valid structure and contains an allowed prefix" to just "has a valid structure". This is necessary to accommodate vendorization, since there is no longer a universal idea of "allowed prefix", but it also means that the validation for DOIs in a given deployment is now much weaker than it was before.

Questions:
1. Is this ok?
2. If it's not, what can we do about it?

In reference to question (2), some initial ideas:
- In a theoretical sense, what we might want is to parameterize the validation routine by including the vendorization variables. That is, instead of fetching a static schema and validating against that, we'd want to "fetch" a "function" that yields a proper "static" schema when supplied with the vendorization variables. In practice, this seems like a bad idea for a few reasons (e.g., we lose the utter simplicity of using published, static schema files).
- Could we instead layer validating against the static, generalized schema with a vendor-specific, Pydantic based validation? Is this something that is already done?

(Context: this came out of a discussion I had with @jjnesbitt about resolving https://github.com/dandi/dandi-archive/issues/2687.)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.