AllenNeuralDynamics / AllenNeuralDynamics/aind-metadata-queries
store queries in flat structure as yaml and add loaders
- Lingua principale
- Python
- Stelle
- 0
- Fork
- 0
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
# User story
As a user, I want to be able to read the docs on a query through a readme associated with a query
*Ideally, this is in the issue title, but if not, you can put it here. If so, delete this section.*
## Library structure
```
query_library/
├── queries/
│ ├── find_subjects_by_cohort.yaml
│ ├── physiology_processed_across_sessions.yaml
│ └── ...
├── query_library/
│ ├── __init__.py
│ ├── loader.py
│ └── generate_query.py
├── scripts/
│ └── generate_helpers.py
├── pyproject.toml
└── tests/
```
As a user, I want to import `aind-data-access-api` and use a module to generate queries which will load queries from the `queries` folder
## Use case
```
from data_access_api import MetadataDbClient
from query_library import generate_query
API_GATEWAY_HOST = "api.allenneuraldynamics.org"
DATABASE = "metadata_index"
COLLECTION = "data_assets"
docdb_api_client = MetadataDbClient(
host=API_GATEWAY_HOST,
database=DATABASE,
collection=COLLECTION,
)
query = generate_query.physiology_processed_across_sessions(
subject_ids=["sub-001", "sub-002"],
min_date=datetime(2024, 1, 1)
)
results = docdb_api_client(query=query, projection={}
```
## Helper
```
# query_library/generate_query.py
from datetime import datetime
from typing import Optional
from query_library.loader import load_and_build_query
def physiology_processed_across_sessions(
subject_ids: list[str],
min_date: Optional[datetime] = None
) -> dict:
return load_and_build_query(
"physiology_processed_across_sessions",
subject_ids=subject_ids,
min_date=min_date
)
def find_subjects_by_cohort(cohort_id: str) -> dict:
return load_and_build_query(
"find_subjects_by_cohort",
cohort_id=cohort_id
)
```
## loader
```
# query_library/loader.py
import yaml
from pathlib import Path
def load_query(name: str) -> dict:
"""Load a query definition from YAML."""
query_dir = Path(__file__).parent.parent / "queries"
path = query_dir / f"{name}.yaml"
with open(path) as f:
return yaml.safe_load(f)
def load_all_queries() -> dict[str, dict]:
"""Load all query definitions."""
query_dir = Path(__file__).parent.parent / "queries"
queries = {}
for path in query_dir.glob("*.yaml"):
with open(path) as f:
query = yaml.safe_load(f)
queries[query["name"]] = query
return queries
```
## generate helpers
Can have a script that loads all the yaml files at instantiation (we can discuss design). it would be nice to have a good interface between the queries and aind-data-access-api
@seanmcculloch - let me know what you think
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.