Support multiple versions of target dandischema
- Dominant language
- Python
- Stars
- 28
- Forks
- 37
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 9
Description
Inspired by
- https://github.com/dandi/dandi-cli/issues/1463
by @kabilar , and ongoing work on "vendorization" of DANDI archive codebase to support multiple instances of DANDI.
ATM both `dandi-cli` and `dandi-archive` re-use `dandischema` Python library for manipulating and validating metadata records. As a result, any upgrade of `dandischema` in server requires upgrade of `dandischema` in `dandi-cli` installation which also might require upgrade of `dandi-cli` itself.
With multiple archive instances in play (already DANDI and LINC and EMBER is coming), synchronizing all of them to align with most recent `dandischema` would be getting close to infeasible. So I think we should distill a solution which would allow `dandi-cli` to manipulate different/needed versions of dandischema.
Possible roadmap to achieve that
- migrate pydantic model to linkml (effort ongoing by @candleindark ) + potentially additional "instance specific" restrictions (TODO: link issues in dandi-schema)
- export linkml models into https://github.com/dandi/schema/tree/master/releases
- define `dandischema_schema_version` -- as long as it is "compatible" in terms of its formulation with `dandi-cli`. In a similar situation in BIDS we have `BIDS_VERSION` and its `SCHEMA_VERSION`, see https://github.com/bids-standard/bids-specification/tree/master/src/schema#version-of-the-schema
- in `dandi-cli` implement loading of models from `linkml` models for the version desired by the server:
- as long as that `linkml` model is described in compatible version of schema (based on minimal `dandischema_schema_version` supported by dandi-cli) , we should be good
- potentially extend the model with instance specific models where needed for validation (most likely the base model would be sufficient to describe)
- not yet sure what we should do for currently used pydantic models in Python interfaces, might need to populate classes for different versions. So issue similarish to what we discussed with @rly and other nwb developers about nwb versions.
Also attn @jwodder @satra @bendichter @dandi/dandiarchive to potentially also think about enhancement to above plan or just to suggest alternative approaches.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.