AllenNeuralDynamics / AllenNeuralDynamics/aind-metadata-queries

store queries in flat structure as yaml and add loaders

Offen
#6 1 Kommentar 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
0
Forks
0
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

# User story
As a user, I want to be able to read the docs on a query through a readme associated with a query
*Ideally, this is in the issue title, but if not, you can put it here. If so, delete this section.*

## Library structure
```
query_library/
├── queries/
│ ├── find_subjects_by_cohort.yaml
│ ├── physiology_processed_across_sessions.yaml
│ └── ...
├── query_library/
│ ├── __init__.py
│ ├── loader.py
│ └── generate_query.py
├── scripts/
│ └── generate_helpers.py
├── pyproject.toml
└── tests/
```

As a user, I want to import `aind-data-access-api` and use a module to generate queries which will load queries from the `queries` folder

## Use case
```
from data_access_api import MetadataDbClient
from query_library import generate_query

API_GATEWAY_HOST = "api.allenneuraldynamics.org"
DATABASE = "metadata_index"
COLLECTION = "data_assets"

docdb_api_client = MetadataDbClient(
host=API_GATEWAY_HOST,
database=DATABASE,
collection=COLLECTION,
)
query = generate_query.physiology_processed_across_sessions(
subject_ids=["sub-001", "sub-002"],
min_date=datetime(2024, 1, 1)
)

results = docdb_api_client(query=query, projection={}
```

## Helper
```
# query_library/generate_query.py

from datetime import datetime
from typing import Optional
from query_library.loader import load_and_build_query

def physiology_processed_across_sessions(
subject_ids: list[str],
min_date: Optional[datetime] = None
) -> dict:
return load_and_build_query(
"physiology_processed_across_sessions",
subject_ids=subject_ids,
min_date=min_date
)

def find_subjects_by_cohort(cohort_id: str) -> dict:
return load_and_build_query(
"find_subjects_by_cohort",
cohort_id=cohort_id
)
```

## loader
```
# query_library/loader.py

import yaml
from pathlib import Path

def load_query(name: str) -> dict:
"""Load a query definition from YAML."""
query_dir = Path(__file__).parent.parent / "queries"
path = query_dir / f"{name}.yaml"
with open(path) as f:
return yaml.safe_load(f)

def load_all_queries() -> dict[str, dict]:
"""Load all query definitions."""
query_dir = Path(__file__).parent.parent / "queries"
queries = {}
for path in query_dir.glob("*.yaml"):
with open(path) as f:
query = yaml.safe_load(f)
queries[query["name"]] = query
return queries
```

## generate helpers
Can have a script that loads all the yaml files at instantiation (we can discuss design). it would be nice to have a good interface between the queries and aind-data-access-api

@seanmcculloch - let me know what you think

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.