Get LinkML-derived Dandiset JSON Schema working with Meditor / dandi-archive frontend
- Dominant language
- Python
- Stars
- 7
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
## Goal
Get the Dandiset JSON Schema generated from the auto-translated LinkML schema (`dandischema/models_linkml/dandiset.json` on the `linkml-auto-converted` branch) working with the [Meditor](https://github.com/dandi/dandi-archive/tree/master/web/src/components/Meditor) metadata editor in `dandi-archive` — i.e. behaving the same way the current Pydantic-generated schema does in production at https://dandiarchive.org/.
When the LinkML-derived schema is served to the frontend in place of the Pydantic-generated one, several gaps are visible (extra/missing tabs and fields, mismatched titles, broken `ADD ITEM` flows on the discriminated-union tabs, etc.). The remaining gaps need fixes either in `pydantic2linkml` (which auto-translates `dandischema.models` → `models.yaml`), in upstream `linkml`'s JSON Schema generator, or in the `dandi-archive` frontend itself. This issue tracks those.
## Tasks
### `dandi-archive` frontend
- [ ] **Adjust the `dandi-archive` frontend (Meditor) to render the LinkML-derived Dandiset JSON Schema.** *(in progress)*
Even with all the schema-side fixes below in place, a few LinkML-vs-Pydantic shape differences need accommodation in the consumer: nullable fields are emitted as `type: [X, "null"]` unions instead of a single-string `type` (Meditor's basic/complex/array/object classification has to recognize these); some `$defs` are recursive (`PropertyValue.valueReference → PropertyValue`) and break a JSON-stringify-based deep copy and Vue's reactivity proxy if dereferenced into JS cycles; the LinkML schema declares `$schema: ".../draft/2019-09/schema"` (Ajv 2020 trips on the unfamiliar dialect) and a top-level `$id` (Ajv complains about duplicate id when the basic and complex slices both retain it); regex `pattern` values use Python-flavored syntax (`\Z`, `{,N}`) that JS `RegExp` can't compile; and the `pydantic2linkml` "Unions as ranges" encoding produces `{$ref: "#/$defs/Any", anyOf: [...]}` which after dereference becomes an over-specified hybrid that breaks `json-layout`'s fresh-form compile (e.g. `ADD ITEM`). Lives on a working branch of `dandi/dandi-archive`; will be tracked separately by a PR in that repo and surfaced here.
### Titles
- [ ] **Improve `pydantic2linkml` to emit missing/correct titles for Dandiset slots.**
After regenerating with the title-emitting `gen-json-schema --title-from title`, only 6 of the 25 top-level Dandiset properties have the right title; 17 have no title and 2 (`name`, `contributor`) inherit a generic title from `CommonModel.slot_usage` (`Title`, `Contributors`) instead of the Dandiset-specific one (`Dandiset title`, `Dandiset contributors`). The two mismatches are flagged directly in `dandischema/models.yaml` by the auto-generated note `pydantic2linkml: Impossible to generate slot usage entry for the {contributor,name} slot. The slot representation … has changes in value in constraint meta slots: ['required'] .` — `pydantic2linkml` should emit a Dandiset-level `slot_usage` override even when the only differing constraint is `required`, so the per-class title carries across.
### Read-only fields
- [x] **Submit upstream PR https://github.com/dandi/pydantic2linkml/pull/69 to `dandi/pydantic2linkml`** so that it translates the `readOnly` JSON Schema keyword expressed in the `json_schema_extra` param of a Pydantic field to the `readonly` meta slot of a LinkML slot definition.
- [x] **Submit upstream PR to `linkml/jsonschemagen.py` to emit `readOnly`.** (Filed an issue https://github.com/linkml/linkml/issues/3528 and submitted a PR https://github.com/linkml/linkml/pull/3537 for the proposed change)
LinkML's metamodel has a `readonly` slot field, but `linkml/generators/jsonschemagen.py` doesn't translate it into the JSON Schema `readOnly` keyword (only `summarygen.py` reads it today). Two-line fix analogous to the existing `prop.add_keyword("title", slot.title)` block. Required so that fields the Pydantic model marks as read-only (`id`, `schemaVersion`, `access`, `url`, `repository`, `identifier`, `dateCreated`, `dateModified`, `citation`, `assetsSummary`, `manifestLocation`, `version`) can be hidden from Meditor — Meditor uses VJSF's `readOnlyPropertiesMode: 'hide'` to suppress them. Without this, Meditor's General tab gets cluttered with read-only metadata and the `access` / `AssetsSummary` sub-tabs appear when they shouldn't.
Verified with LinkML 1.11.0 that the `readonly` slot is silently dropped by `gen-json-schema` — it does not translate to JSON Schema's `readOnly` keyword (nor to any annotation, comment, or other marker). The property for `bar` (with `readonly`) is indistinguishable from `baz` (without).
test_readonly.yaml
```yaml
id: https://example.org/test
name: test
prefixes:
linkml: https://w3id.org/linkml/
default_prefix: test
imports:
- linkml:types
classes:
Foo:
attributes:
bar:
range: string
readonly: "this field is read-only"
baz:
range: string
```
gen-json-schema --title-from title test_readonly.yaml
```json
{
"$defs": {
"Foo": {
"additionalProperties": false,
"description": "",
"properties": {
"bar": {
"type": [
"string",
"null"
]
},
"baz": {
"type": [
"string",
"null"
]
}
},
"title": "Foo",
"type": "object"
}
},
"$id": "https://example.org/test",
"$schema": "https://json-schema.org/draft/2019-09/schema",
"additionalProperties": true,
"metamodel_version": "1.11.0",
"title": "test",
"type": "object",
"version": null
}
```
### Discriminated unions
- [ ] **Emit `discriminator` + `oneOf` for Pydantic discriminated unions in `pydantic2linkml` (and `linkml/jsonschemagen`).**
Pydantic discriminated unions (`Annotated[Union[Person, Organization], Field(discriminator="schemaKey")]`) emit JSON Schema `oneOf + discriminator{propertyName: "schemaKey", mapping: {...}}`, which Meditor uses both to render a "pick the variant" control on `ADD ITEM` and to auto-populate `schemaKey` so the new item validates. The LinkML translation currently emits `any_of` slot ranges that round-trip to `{$ref: "#/$defs/Any", anyOf: [...]}` — no discriminator. Result: clicking `ADD ITEM` on the `Dandiset contributors` and `Subject matter of the dataset` tabs creates an empty record that fails validation with `must match a schema in "anyOf"`. Both `pydantic2linkml` (preserve the discriminator info) and `linkml/jsonschemagen` (emit `discriminator` + `oneOf` from that info) need changes.
- [ ] **Emit JSON Schema `const` for fixed-value fields like `schemaKey`.**
Pydantic emits `Literal["Person"]` fields as `{"const": "Person", "default": "Person", ...}`. The LinkML translation emits single-value enums (`{"enum": ["Person"]}`) instead. This affects two places in Meditor: (1) the top-level `Dandiset.schemaKey` is auto-hidden by VJSF when emitted as `const` but renders as a one-option dropdown when emitted as a single-element `enum`; (2) on discriminated-union variants (`Person.schemaKey`, `Organization.schemaKey`, etc.), `const + default` lets VJSF auto-fill the discriminator value when `ADD ITEM` creates a new item — without it, `ADD ITEM + SAVE ITEM` fails validation because the new record has no `schemaKey`.
- [ ] **Honor Pydantic `slot_usage` range narrowing (e.g. `Dandiset.wasGeneratedBy → Project`).**
In `dandischema.models`, `Dandiset.wasGeneratedBy` is narrowed to `List[Project]`; the parent `CommonModel` exposes the broader Activity-family. Pydantic's JSON Schema reflects the narrow form (`items = $ref: Project`); the LinkML translation keeps the parent's polymorphic union (`anyOf: [Activity, Project, PublishActivity, Session]`). The auto-generated note `pydantic2linkml: Impossible to generate slot usage entry for the wasGeneratedBy slot. … changes in value in constraint meta slots: ['range'] .` confirms `pydantic2linkml` bailed on emitting the override. It should produce the per-class `slot_usage` entry with the narrowed `range` even when the diff is in a "constraint meta slot." Without this, `Associated projects` → `ADD ITEM` fails the same way `contributor`/`about` do.
### Property ordering
- [ ] **Investigate Dandiset JSON Schema property ordering.**
LinkML's `gen-json-schema` emits top-level Dandiset `properties` alphabetically, while Pydantic preserves Python declaration order. Visible in Meditor as: tabs ordered `Subject matter of the dataset → Dandiset contributors` instead of production's `Dandiset contributors → Subject matter of the dataset`, and the General tab fields ordered `Acknowledgement, Description, …` instead of `Dandiset title, Description, Study Target, License, Protocol, Keywords, Acknowledgement`. Cosmetic only — but worth resolving so the LinkML-driven UI matches production. The fix likely lives upstream (have `linkml` honor the slot list order declared on the class).
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing dandischema/models_linkml/dandiset.json and dandischema/models.yaml with the production Pydantic-generated schema, then choose one unchecked gap. Read the relevant pydantic2linkml translation or linkml/jsonschemagen.py entry point, and use the dandi-archive Meditor behavior as the check; done means the LinkML-derived schema matches production for that case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- developer-experience, full-stack, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100