AllenNeuralDynamics / AllenNeuralDynamics/aind-metadata-queries

store queries in flat structure as yaml and add loaders

Abierto
#6 1 comentario 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Python
Estrellas
0
Forks
0
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

# User story
As a user, I want to be able to read the docs on a query through a readme associated with a query
*Ideally, this is in the issue title, but if not, you can put it here. If so, delete this section.*

## Library structure
```
query_library/
├── queries/
│ ├── find_subjects_by_cohort.yaml
│ ├── physiology_processed_across_sessions.yaml
│ └── ...
├── query_library/
│ ├── __init__.py
│ ├── loader.py
│ └── generate_query.py
├── scripts/
│ └── generate_helpers.py
├── pyproject.toml
└── tests/
```

As a user, I want to import `aind-data-access-api` and use a module to generate queries which will load queries from the `queries` folder

## Use case
```
from data_access_api import MetadataDbClient
from query_library import generate_query

API_GATEWAY_HOST = "api.allenneuraldynamics.org"
DATABASE = "metadata_index"
COLLECTION = "data_assets"

docdb_api_client = MetadataDbClient(
host=API_GATEWAY_HOST,
database=DATABASE,
collection=COLLECTION,
)
query = generate_query.physiology_processed_across_sessions(
subject_ids=["sub-001", "sub-002"],
min_date=datetime(2024, 1, 1)
)

results = docdb_api_client(query=query, projection={}
```

## Helper
```
# query_library/generate_query.py

from datetime import datetime
from typing import Optional
from query_library.loader import load_and_build_query

def physiology_processed_across_sessions(
subject_ids: list[str],
min_date: Optional[datetime] = None
) -> dict:
return load_and_build_query(
"physiology_processed_across_sessions",
subject_ids=subject_ids,
min_date=min_date
)

def find_subjects_by_cohort(cohort_id: str) -> dict:
return load_and_build_query(
"find_subjects_by_cohort",
cohort_id=cohort_id
)
```

## loader
```
# query_library/loader.py

import yaml
from pathlib import Path

def load_query(name: str) -> dict:
"""Load a query definition from YAML."""
query_dir = Path(__file__).parent.parent / "queries"
path = query_dir / f"{name}.yaml"
with open(path) as f:
return yaml.safe_load(f)

def load_all_queries() -> dict[str, dict]:
"""Load all query definitions."""
query_dir = Path(__file__).parent.parent / "queries"
queries = {}
for path in query_dir.glob("*.yaml"):
with open(path) as f:
query = yaml.safe_load(f)
queries[query["name"]] = query
return queries
```

## generate helpers
Can have a script that loads all the yaml files at instantiation (we can discuss design). it would be nice to have a good interface between the queries and aind-data-access-api

@seanmcculloch - let me know what you think

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.