AssetsSummary: numberOfSamples / numberOfCells under-count when the same id appears across subjects
- Dominant language
- Python
- Stars
- 7
- Forks
- 15
- PR merge metrics
- No merged PRs in 30d
Description
While adding `numberOfSessions` in #409, I noticed the aggregator has the same latent issue for samples (and probably cells/slices) that the session work is fixing:
`stats["tissuesample"]`, `stats["cell"]`, and `stats["slice"]` are flat lists of identifier strings, deduplicated globally rather than per-subject. So if `sub-01` and `sub-02` both have a `sample-1` (or both reference a `BioSample` with the same `identifier`), the aggregator collapses them into a single entry and `numberOfSamples` is off by one.
Relevant code:
- Filename-derived samples: `dandischema/metadata.py:537` (the `("sample", "sample-", "tissuesample")` entry in the entity loop) appends to `stats["tissuesample"]` keyed only on the sample id.
- `wasDerivedFrom` / `BioSample`-derived samples: `dandischema/metadata.py:471` (`_get_samples`) and `:522-527`. Identifiers go through `sanitize_value(value["identifier"])` and then dedupe globally per `sampleType` (`cell`, `slice`, `tissuesample`).
- Final tally: `dandischema/metadata.py:589` (`numberOfSamples = len(tissuesample) + len(slice)`) and `:591` (`numberOfCells = len(cell)`).
The cell/slice path is less likely to bite in practice because BioSample identifiers are typically globally unique, but the filename `sample-*` path is much more likely to collide across subjects.
Proposed fix: key on `(subject, sample)` / `(subject, cell)` / `(subject, slice)` pairs the same way #409 keys on `(subject, session)`, then take `len(unique_pairs)`.
Caveats:
- This changes the value of existing fields (`numberOfSamples`, `numberOfCells`) for any dandiset where the same id appears under multiple subjects. That's a semantic shift, so it should land deliberately and probably be called out in the changelog.
- For the `_get_samples` path, the BioSample `identifier` is supposed to be globally unique. If we move to `(subject, sample-id)` keying there, the subject context needs to be plumbed through `_get_samples`.
Spun out of #409 review discussion.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.