dandi / dandi/dandi-schema

Add dqv:QualityMeasurement and overall vocab-dqv to describe quality of dandisets (or individual files?)

Open
#424 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7
Forks
15
PR merge metrics
No merged PRs in 30d

Description

Prompted by discussion with @dandi/ember today and looking at https://github.com/ESIPFed/science-on-schema.org/blob/main/guides/Dataset.md lacking any quality descriptors, claude pointed out to https://www.w3.org/TR/vocab-dqv/

full claude answer

This is a good question, and it gets at a real gap. Let me verify a couple of specifics about schema.org's current vocabulary before answering, since some quality-relevant terms have been added relatively recently.Short answer: schema.org proper has essentially no first-class notion of dataset "quality." It deliberately stops at descriptive/discovery metadata, so there's no `schema:quality`, no quality dimensions, and — notably — not even a native `schema:conformsTo`. That last one has been an open request since 2017 (issue #1516, "Allow Dataset to indicate according to specification URL," which has gone quiet). So anything quality-related is either an oblique reuse of a general property or a matter of pulling in a purpose-built vocabulary via the JSON-LD context — exactly the way the SOSO guide already mixes in `prov:`, `spdx:`, `time:`, and `gstime:`.

The oblique native hooks, roughly in order of how defensible they are:

- **Provenance (PROV-O / ProvONE).** Already recommended by SOSO. For scientific data this is arguably the strongest quality signal schema.org-land offers, because trustworthiness and reproducibility are quality — `prov:wasDerivedFrom`, `prov:wasGeneratedBy`, plus `schema:measurementTechnique` on variables.
- **`schema:additionalProperty` → `PropertyValue`.** The generic escape hatch SOSO uses for spatial reference systems. You can hang arbitrary named metrics here (percent-missing, valid-record count, etc.). Cheap and works, but the semantics are opaque unless `propertyID` resolves to a real metric definition.
- **`schema:hasMeasurement` → `QuantitativeValue`.** A newer, unit-aware variant of the same escape-hatch idea.
- **`schema:review` / `schema:aggregateRating`.** Dataset is a `CreativeWork`, so these apply. Good for curatorial/subjective quality ("reviewed, 4/5"), and `reviewAspect` lets you name what's being rated — but it's human-judgment framing, not machine metrics.
- **`schema:Certification`.** Exists, but it's mapped from the GS1 vocabulary and aimed at product/organization/person certifications — ISO, organic, energy labels. You *could* point its `about` at a Dataset to assert "passed certification X," but no dataset consumer expects it there, so it's a stretch.
- **`dct:conformsTo`** (Dublin Core, not schema.org). This is the natural "does it conform to spec X" hook, and it's what MLCommons **Croissant** does — Croissant datasets declare conformance by including `dct:conformsTo` at the dataset level. For your world this is the obvious place to assert BIDS-version conformance.

The real answer, if you want structured, machine-actionable quality: layer **W3C DQV (Data Quality Vocabulary)** — https://www.w3.org/TR/vocab-dqv/ — on top. It's the 2016 WG Note designed for precisely this, built to sit alongside DCAT (https://www.w3.org/TR/vocab-dcat-3/), which in turn is where schema.org's Dataset came from. DQV gives you `dqv:QualityMeasurement` (a metric applied to a distribution with a value), `dqv:Metric`, `dqv:Dimension`, `dqv:Category`, and `dqv:QualityAnnotation` (for feedback or conformance certificates). So "94% of records complete" becomes a measurement of a completeness-dimension metric, not a nameless number.

A minimal layered sketch:

```json
{
"@context": [
"https://schema.org/",
{ "dqv": "http://www.w3.org/ns/dqv#", "dct": "http://purl.org/dc/terms/" }
],
"@type": "Dataset",
"name": "...",
"dct:conformsTo": { "@id": "https://bids-specification.readthedocs.io/en/v1.9.0/" },
"dqv:hasQualityMeasurement": {
"@type": "dqv:QualityMeasurement",
"dqv:isMeasurementOf": { "@id": "https://example.org/metrics/bids-validator-pass" },
"dqv:value": true
}
}
```

For BIDS specifically the mapping falls out cleanly: `dct:conformsTo` → the spec version, a DQV measurement (or a plain `PropertyValue`) for "passes `bids-validator`," and PROV for derivatives. That's essentially reframing validation-as-conformance in RDF terms, which is probably the most honest reading of "quality" you can attach here.

One caveat worth stating plainly: Google Dataset Search — the dominant consumer of schema.org Dataset markup — indexes discovery fields and will do nothing with DQV, `dct:conformsTo`, or provenance. So any quality markup is for specialized harvesters and your own tooling (DataLad/DANDI-side), not for rich-result eligibility. Given that, the pragmatic question is whether you'd rather emit DQV for interoperability with the DCAT/data-quality ecosystem, or just stash the same facts in `additionalProperty` for lighter weight. Happy to sketch a fuller DQV example with dimensions/categories if you want to see how far it stretches for neuroimaging conformance.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the linked science-on-schema.org Dataset.md guidance and the W3C vocab-dqv specification, then inspect the existing dandi-schema definitions. Clarify whether measurements apply to whole dandisets, individual files, or both; done means the intended DQV vocabulary and scope are represented consistently in the schema.

Written by the indexing model from the issue text.

Assessment

Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.