Overhaul models so that "unfinished" metadata can be represented without cheating Pydantic
- Dominant language
- Python
- Stars
- 7
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
**Note:** @satra filed a spiritually-identical issue at almost the same time: #204
---
(This is an accumulation of various things discussed elsewhere which need to be written down.)
Currently, it is possible for users of the Dandi Archive API to submit asset & Dandiset metadata that does not fully conform to the relevant dandischema model, and the Archive will accept, store, and return such flawed metadata, largely via use of `DandiBaseModel.unvalidated` (being replaced by Pydantic's `construct` in #203). I believe part of the motivation for this is so that web users can fill in metadata over multiple sessions without having to fill in every field in a single sitting.
This results in API requests for asset & Dandiset metadata sometimes returning values that do not validate under dandischema's models; in particular, if a user of dandi-cli's Python API calls `get_metadata()` instead of `get_raw_metadata()`, the call may fail because our API returned metadata that doesn't conform to our own models (See dandi/dandi-cli#1205 and
dandi/dandi-cli#1363).
The dandischema models should therefore be overhauled as follows:
* There should exist models for representing Dandiset & asset metadata in a draft/unfinished state. These models should accept all inputs that we want to accept from users (both via the API and the web UI), store in the database, and return in API responses. (It is likely that such models will have all of their fields marked optional aside from the absolute bare minimum required.)
* The `get_metadata()` methods of dandi-cli's Python API should return instances of these models.
* There should exist functionality for determining whether an instance of a draft/unfinished model meets all of the requirements for Dandiset publication.
* One possible way to implement this would be to have separate models for published metadata that inherit from the draft models and make various fields non-optional.
CC @satra @dandi/dandiarchive @dandi/dandi-cli
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.