dandi / dandi/dandi-cli

[question] expose programmatic access to dandiset metadata?

Open
#1,361 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
28
Forks
37
Avg merge
1d 17h
Merged PRs (30d)
9

Description

hello again :)

question for y'all, gauging interest in a PR - looking through the docs and the source to see the best way to query dandiset metadata and having trouble finding the "right way" that it's supposed to be done.

It seems like this package is primarily oriented around the click cli commands, and the internal functions aren't necessarily expected to be used except in 'advanced' cases - eg. `dandi.download.download` doesn't have a docstring, etc. I can't tell if y'all intend this to be an SDK for downstream packages to build off, but it would be nice to be able to do that!

So it seems like to get the dandiset metadata that one would do
```bash
dandi download --download dandiset.yaml
```
and then parse the resulting yaml file.

That's not terribly convenient for programmatic use, where I might want to do
```python
from dandi import get_metadata
meta = get_metadata('DANDI:ID') # type: dandischema.models.Dandiset
```

I see one previous issue ( https://github.com/dandi/dandi-cli/issues/1205 ) that shows what might be the recommended method:

```python
from dandi.dandiapi import DandiAPIClient

client = DandiAPIClient()
dandisets = list(client.get_dandisets())

# this works
dandisets[-3].get_metadata()
```

and it seems like this also works:

```python
from dandi.dandiarchive import parse_dandi_url

url = parse_dandi_url('DANDI:0000N')
dset = url.get_dandiset(url.get_client()) # type: dandi.dandiapi.RemoteDandiset
meta = dset.get_metadata() # type: dandischema.models.Dandiset
```

so I'm curious if I can help with either some docs or some helper functions, let me know which, if any of these yall would be interested to have in a PR:

- Documenting how to get metadata programmatically using the above syntax - this could go in an 'examples' or 'guide' directory that we might also leave some dangling stubs to entertain future documentation PRs?
- Documenting some of the surrounding objects and how they're used - there are already docs for some of the relevant classes (eg. https://dandi.readthedocs.io/en/latest/modref/dandiarchive.html#dandi.dandiarchive.ParsedDandiURL ) but I wouldn't really know how to use them if not for reading the source code. eg. it's not altogether obvious that a `DandisetURL` would yield a `RemoteDandiset` which can `get_metadata`, but it might be more obvious with some high-level description of how these objects are intended to be used
- Convenience function for `get_metadata` that wraps the above steps - not sure how applicable this would be to the more general download function, since typically that will download a series of files, but at least for the metadata it seems like this would be a common want / wouldn't incur any additional maintenance burden since it just wraps existing code.

This also could be handled at the level of the web API docs, which I couldn't find (but might be around here somewhere!) eg. rather than using the python API at all, one could just `GET https://api.dandiarchive.org/api/dandisets/000540/versions/0.230515.0530/` for example, so that might be another option but out of scope for this repo.

anyway lmk what would be useful, or if i missed something!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.